A deep neural network trained by backpropagation will converge to a local minimum if you use gradient descent or even stochastic gradient descent. However, there are many components added to a deep neural network like dropout and batch normalization, which as far as I know, do not come with convergence guarantees.
There are no guarantees about finding a global minimum
jostmey t1_j4vmqcs wrote
Reply to [D] Are there any results on convergence guarantees when optimizing NNs? by Dartagnjan
A deep neural network trained by backpropagation will converge to a local minimum if you use gradient descent or even stochastic gradient descent. However, there are many components added to a deep neural network like dropout and batch normalization, which as far as I know, do not come with convergence guarantees.
There are no guarantees about finding a global minimum