Heat pumps have an efficiency curve that is directly tied to the outdoor air temperature. As the temp drops, so does the efficiency. At some point, the efficiency of the heat pump will drop far enough that the comparable cost to heat with gas or oil is actually lower.

I recommend anyone heating with multiple heat sources set up a quick and dirty spreadsheet to show you what your crossover temperature is. The efficiency curves for heat pump outdoor units are published by the manufacturer, as are all the other values you'll actually need to create the spreadsheet.