While in practice this is true, a lot of work is being done to try to figure out how to make shallow neural networks train as fast as deep ones. We know by the UAT that in principle a shallow network with similar paramaters counts to a deep one should be capable of learning the same decision boundary...
It depends on the function that you are trying to approximate.
I can give you a function that a shallow nnet would approximate better and functions that deep nets approximate exponentially better even with one more layer (in terms of number of neurons n). In the limit n->\infty, both reach arbitrary small errors (obviously often with different number of parameters).
https://en.m.wikipedia.org/wiki/Universal_approximation_theo...