I'd probably lean more toward Back Bay, as there's more "stuff" to walk to and it's just a cool area to walk around. Seaport is nice but may be a bit sterile for some folks (and you've probably seen some of the restaurants in the airport on the way here.)