So, there’s a trick where you write a differentiable program and swap out expensive bits with a neural network, which I think is probably related to this. Looking at the article, I think you would very quickly run into some hard problems in differential geometry if you tried to make this formal.
Diffeologician t1_j45j7c4 wrote
Reply to comment by cdsmith in [D] What's your opinion on "neurocompositional computing"? (Microsoft paper from April 2022) by currentscurrents
So, there’s a trick where you write a differentiable program and swap out expensive bits with a neural network, which I think is probably related to this. Looking at the article, I think you would very quickly run into some hard problems in differential geometry if you tried to make this formal.