In addition, that inertia isn't always a good thing on part of the person getting hit.

An often overlooked factor in how damaged you get is the amount of force it takes to overcome inertia. Hit a smaller person in the body, they go flying without breaking ribs because the amount of force necessary to knock them over is less than what's required to break ribs. Hit a bigger person, and their body has to soak more impact before movement occurs. Depending on the size difference and the amount of tissue padding the larger person has, there is a wide range of potential outcomes.

This combines unfavorably with the kinetic force equation, as velocity has more effect on force than mass. Smaller object moving faster = more penetration, which is bad for the person getting hit. It's far more likely to break ribs/jaws/orbitals/teeth as the smaller bits will fail before inertia is overcome.

Training helps with this, as you learn when to strike for maximum effect. It's counterintuitive due to the desire to avoid pain, but it's best to hit someone that is coming straight at you (who is usually swinging for your face) because you're adding their mass & velocity to your own. Hitting someone that's retreating reduces the force of your strikes. Think of the difference between a head-on collision and getting rear ended by someone going 50 when you're going 35.

For example: putting strong kicks into someone my size is usually less useful than putting them into someone larger than me. I have to time my kick so the person my size is advancing into the strike, giving me more force to overcome (and therefore more force I can put into them before they start changing direction) which directly translates into damage inflicted. With a larger opponent, I can deliver a full-force kick to the abdomen and they will fold like an omlette before they go flying because all the energy of my kick went straight into their body instead of getting wasted.