So the paper which talks about the Spline theory of DL says that even in latent representation NN are incapable of interpolation and that's a very important thing to know about. If we know this then we can design loss functions that works to better understand the global manifold structures.