There was a really good paper a few years ago that identifies some biases in how DNNs learn might explain why they work so well in practice as compared to alternatives. Essentially they are biased towards smoother solutions, which is often what is wanted.

This is still an area of active research, though. I think it's fair to say we still don't quite know why DNNs work as well as they do.