This is the bullshit logic that results in answers like 'compete under a neutral flag'.

If that's actually how it worked, you'd be right. But the Olympics are actually a system of a country showing how impressive they are - especially when you're talking about the top few countries - and nobody fucking believes that the athletes aren't Russian just because they don't have a Russian flag on.

You punish the athletes because the athletes are competing under the auspices of a government. The same reason you punish the people of Russia (or wherever) through economic sanctions even though those don't directly impact those in power.