We have a similar room, which is essentially a porch that has been converted into a room.

We were on a tight budget, so we thoroughly caulked all sides of the baseboards, electrical outlets, and around the window frames, put in a thick carpet and padding on the floor, heavy fabric drapes for the windows in winter (but we change them out in summer), and we use the plastic film and a humidifier in winter as well.

This helped a lot, but of course isn't enough when the temp drops below 20°F. At that point we use a 900 watt space heater, but it's only needed sparingly. That's the least expensive route, anyway.