Have you ever worked in the private sector? Because they can't handle those things. The belief they can is based on the false idea that people are basically good, honest, and put the public interest above their own greed. That's not how things actually work though.

The incentives are all wrong for the private sector to even attempt to handle those things. They prioritize short term gains and don't consider the long term costs. Government is required to ensure companies don't risk public safety for short term gain, when the incentives set by capitalism encourage them to do so.

Yes, I support the health department inspecting restaurants, if the private sector tried to do that the restaurant owners would pay them to pass even when they should fail. The government can do that effectively because they aren't trying to make money, so they don't have a motive to pass a restaurant that should fail.

There are many things society needs to function where the incentive for profit encourages people to the opposite of what needs to be done. That's where government comes in. In a property functioning capitalist system the government tries to align the interest of the private sector with the public good.

Profit motive and public good aren't aligned by magic as you seem to think. The government is required to keep those things aligned by setting the rules for the private sector and creating incentives to do the right thing. Where that isn't enough to align profit motive and public good, the government runs things themselves, as they run courts, the military, programs to care for the sick and elderly who can't care for themselves.