It would seem to be so easy to take advantage of all the oxygen in the air but it turns out to generally be extremely not worth the trouble.

The first major problem is just getting the air and making use of it, which is insanely complicated and requires lots of complex machinery. We're talking turbojet, ramjet, or scramjet engines. And while it may seem that a rocket engine is insanely complex, they are actually a lot simpler than jet engines in many regards. The first liquid fueled rocket propelled vehicle flew about 20 years before the first jet powered aircraft, for example. Even more so it's incredibly difficult to design air breathing engines which operate over wide ranges of speeds and altitudes. Consider the SR-71 and the ridiculous level of engineering that went into that and how those speeds and altitudes are just baby steps compared to orbital launch. It takes about a minute and 45 seconds for a Falcon 9 to surpass the speed and altitude of the SR-71. It is very challenging to build an airbreathing engine that would be worth its weight.

Once you add airbreathing to a launch vehicle you would want to spend more time in denser atmosphere in order to maximize its usefulness. However, that's very problematic for several reasons. For one it creates much more drag to spend so much time in denser atmosphere, which saps efficiency. For another it adds more aerodynamic heating and strong aerodynamic forces forcing you to add more heat protection and increase the strength of the vehicle. All of which adds weight, complexity, and potential failure modes.

In contrast, if you just say no to airbreathing at all you end up with a simpler vehicle that only has rocket engines (saving weight and complexity) and you have a much simpler optimization problem for launch. You can climb out of the thick atmosphere early and do the majority of acceleration in vacuum or near vacuum.