I'm in your camp here they screwed up, and the right thing would have been to refund you and everyone go on their way. Whatever admin costs they incurred well that would be on them here, as you did nothing wrong.

You really need a copy of their policies. It should be a part of the application and other paperwork you filled out. The language there determines if this is small claims court or a credit card chargeback (assuming it's credit card).

You should also look up what your state/cities renters protection laws are to see if there is anything there that helps you.

The unfortunate thing is regardless of the outcome here, if you have to take legal or credit card chargeback actions, you probably get blacklisted from this company's rental units, which isn't in your advantage long term but may not be a huge loss either.