You only see that in densely-packed areas, though. I also live in Japan, and have spent a very large amount of time out in Saitama[1], where for the most part, parking is free as well.

Where I live, parking works as you describe.

[1] The prefecture next to Tokyo, for those that aren't deeply immersed in Japanese geography.