Lol I've definitely done this. Was on a trip with fam and I was a bit underdressed than them because I thought we'd just be walking around DC anyway. Got hungry but all restaurants around us were booked -- until we found one with accommodation.

Unfortunately, our hunger got ahead of us and we didn't even bother to check the menu first. It was a fancy ass restaurant. An appetizer costs as much as a main dish at your typical $$ or even $$$ places. Not sure why we didn't leave when we could... maybe we felt bad since the servers were already giving us their best service, and we were already settled in at the middle-ish part of the restaurant.

I couldn't take off my winter jacket or else I'd look like an idiot with a graphic t-shirt with a huge image of ramen. My family was fine and their outfits fit in well despite being casual. I felt crappy the whole dinner, and we were out $250 while not even being full at the end.