Depends on the reason it's a HCOL area to begin with.

DC? No, it will always be HCOL as it's the seat of the federal government.

An oil boom town where the wells have dried up? Yes.

Ignoring those two extremes, yes I think you're right in theory, that less demand should in turn depress prices overall, but that is assuming the people fleeing aren't being replaced by people from an even higher COL area. Or even wealthy people from other countries.