I'm talking about four nines in terms of reliability (not crashes) and on level 3, where the car still has all the regular driving controls - so if there is a problem or the system has high uncertainty about certain road situations it can force/return control to the driver and have them navigate the unknown situation. I'm also probably being too optimistic predicting four nines in 5 years, it'll probably be 10 years or more for people to be able to reliably take their hands off the wheel for extended periods of time on interstates.

Edit: I realize that level 4 doesn't ask you to take over, so I've edited my comments to reflect that.

the issue is that randomly requiring non-professional drivers to take over during a trip is an underestimated danger. people get accustomed to it working 99% of the time and get lazy - then the 1% it fails and the driver is asleep or distracted or watching a movie and becomes a major safety issue.

ex all the horror stories of distracted Tesla drivers using autopilot like full autonomous driving today.

Furthermore, if you stipulate hand-offs will sometimes be needed, you now can't use the car for many of the use cases that proponents want because you need a competent adult driver to take over.

And, yes, people imagine scenarios like remote drivers taking over. That seems a very difficult problem--especially if the situation is such that the autopilot is confused.