It's genuinely not that cold out, so you guys should be fine to do activities that involve some time outside. But also, what do you enjoy doing?

The Gardner museum is awesome (I recommend doing a tour) and the indoor greenhouse atrium is nice to visit during the winter months. the MFA is also a fantastic art museum.

Beacon Hill, the South End, and the public garden, and Back Bay are all beautiful areas you can wander around to enjoy the scenery and pop into cafes/ local shops.

There's great food all over, but knowing what you like and how fancy you want to go would be helpful.

A lot of the local breweries do tours if you're beer people.

Harvard campus / Harvard square is interesting to explore if you haven't been yet. The Harvard art museum and Harvard natural history museums over there are also great.