Cars and roads are built for human reaction times. That's why you have low speed limits in urban areas. You can have a pile of latencies attributable to processing a scene and still have superhuman reaction time that contributes to outperforming humans.

It's analogous to communications latency. High latencies are very annoying to humans, but below a threshold they stop mattering.