Have you done an energy audit? It can be hard to know where your heat is going without one. The basic one seals your door, pumps air into the house and sees where it leaks out. Most utilities have a program where you can get a basic audit for free. This will help you find out what can be done to tighten up the house.

Take a look at your furnace, the last time it was serviced they'll have put a tag on it, see what it's efficiency is. If it's under 80% it's quite likely a new furnace will pay for itself in a relatively short time. We're actually due for that at my house.

Finally you mention new windows. Sadly I see a lot of windows installed poorly. Feel around the edge of your windows and make sure there aren't air leaks. A good installer will have insulated around the windows, either by packing in fiberglass or with spray foam. Otherwise your windows have vents that just suck your warm air out. This is where the energy audit really helps.