A big category of cluster-friendly HPC cluster codes works by running in lockstep on all the nodes. Apparently it works quite well for things like weather forecasting where the problem is naturally divided into a grid but there is still significant communication needed between grid tiles. So it's quite different from memcached type things.

https://www.cs.fsu.edu/~engelen/courses/HPC/Synchronous.pdf