Speaking as someone with social anxiety, and severe general anxiety, a great way to approach this is saying "My bad. I'm sorry."

That's all that needs to be said, and it conveys so much more than the words let on. And it's casual enough that no one feels like you're prostrating yourself for pity, which is a huge fear of mine