I grew up areligious. I highly recommend The Catholic Bible, Personal Study Edition (green cover). Each section is has an introduction which explains the history, the original text, when, why and how subsequent versions changed it. It gives context on why some books were included and others were not which I found really illuminating. It also has copious footnotes.

My favorite part is that it explains common symbols/allegories/stories and how these are interpreted by various groups throughout history with contextualization. Seeing the evolution of how different groups viewed passages overtime is great for literary references because how we view it now is radically different than even 100 years ago.

ETA: KJV, despite people here insisting on it, didn't influence early Americans or Shakespeare. Most pilgrims brought over the Geneva Bible which is the foundation of puritanical Christianity and most American cultural beliefs -- which is why you won't find the same quotes in the KJV.