If you've lived in Boston proper, no, Greenfield is not rough or scary. If you've mostly lived in the wealthier suburbs, then yes, it is more working class. AFAIK the crime is mostly drug related rather than violent crime. I've never felt unsafe anywhere in Greenfield or Turners tbh, and there's a fair bit of art and music going on there as well as down in Amherst/ Northampton and up in Brattleboro, and plenty of nature! It's also gentrifying quickly given the affordability.