The problem is, you’re looking for something that a lot of other people are also looking for and is super popular right now: quiet small town charm with a progressive culture. Thus the demand is driving all the prices in these places up. Even the bad places are getting pricey.

Edit: sorry I’m not all that helpful in answering your question. In hindsight I’m probably just stating the obvious