Back Bay a million times over. Both in terms of accessibility and quality of restaurants, architecture, etc. Seaport is kind of a cookie-cutter Big City Neighborhood^TM, and doesn't offer much that you couldn't get literally anywhere else in a major US city. For better or worse, it's also very much where the Instagram type of crowd hangs out.