Get some insulation board and tighly seal it. It is an easy DIY project. Check around windows and doors and make sure you get some of the trim that goes around the doors and windows to let them close completely.

Our home is new and has blown in insulation, but the unused fireplace was causing at least a 20 degree temp difference despite the damper being closed.