You can train a neural network without backpropagation (with a gradient method) and it will just be slower.

There are plenty of derivative free methods as well.

It’s not clear whether a function being nondifferentiable means anything without further specification since classification is practiced quite successfully in many cases.