Usually it is to increase the receptive field of the network at a given location (more spatial context). Higher resolution features are then recovered via skip connections if necessary

You need to use a program like colmap for sparse scene reconstruction to recover the camera intrisics (focal length, lens distortions) and extrinsics (camera positions and orientations)

I would have the network regress the lens distortion parameters (like k1 to k6, p1, p2). You should be able to produce synthetic rendered training data.