If you're going to be commuting to Annapolis daily, you do not want to live in Hampden, that would be a terrible commute and you'd hit all of the traffic hotspots. Mount Vernon is also probably a bit too far into the city for a comfortable commute to Annapolis, but if you like the neighborhood it might be worth it. Locust Point, similarly, isn't great for an Annapolis commute, in my opinion, since the fastest option involves tolls and it takes a while to get up/down Fort Ave on the way home. You could take the shortcut across the railroad tracks on Andre St. on the way south, but that's not an option on the way north.

If you like having a yard, Brooklyn, the border neighborhood between Baltimore City and Anne Arundel County, is a great location for commuting to Annapolis (as are the parts of AACo. just south of it on Rt 2), but it's a working class suburb, the things within walking distance aren't places I'd recommend to a newcomer to the area, and it's just annoying enough to get into downtown that it feels further from the rest of Baltimore than it is. If you want to have an actual in-Baltimore experience, the southern neighborhoods along 295 or Hanover street are all good options. I happen to think Pigtown is overpriced, but it's very convenient for getting to Annapolis. Look at the house/street layouts and decide what would make the most sense for your transportation, housing, and parking needs.

If you have asthma, Brooklyn and Curtis Bay have relatively poor air quality, due to being between two incinerators, the coal depot, and a variety of other industrial establishments. Air quality is the biggest reason I relocated to another neighborhood from Brooklyn.

If you're used to Norfolk, Baltimore is fine, just be aware of your surroundings, be polite to people, and secure your belongings. Being polite and friendly with your neighbors goes a long way. I've lived and worked in several areas of the city and haven't had any major problems aside from minor property crimes and some odd social interactions.