GPUs are usually faster for inference simply because they have more ALUs/FPUs but they are also less efficient.