Typically, if you're solver is not written in Pytorch/tensorflow itself you can't easily calculate gradients through them as your computational graph doesn't capture the solver. If your soler is also written in the framework and differentiable you might be able to just backpropagate through it though. Otherwise, the Neural ODE paper that was linked here a few times has an adjoint formulation that gives you the gradient wrt to the solver as a solution to another ode, but this is specific to their problem and won't apply to non-differential equations.