Easiest solution is to buy some plastic insulation film for your windows. Also, get a door sweep to plug up any gaps in doors leading outdoors. For people with forced air, check your filters because a dirty filter will cause the furnace to struggle.

But, dropping nearly 10 degrees sounds like something might be wrong with your system. Keep an eye on if it continues to drop.