I really like your last point there. That is a good analogy.

I guess my question boils down to "how to think about information in a trained model". What I am wondering is whether a model can carry more information than it's raw size which I think it may be able to conceptually as the relationship between neurons carries information but isnt reflected in the file size of a model.

So like a regression represents a point cloud, could we now vectorise a book or a movie (if that was what we wanted)?