Check to see exactly which gree you have. In Massachusetts, roughly half your heating load is for when the temperature is above 27°, half below. So if I look, the only Gree 24k I found that fits the 23seer, 10.5 HSPF is their multi+plus unit, which at 27° has a COP of 1.76 at full capacity. They don't list their partial capacities, assuming your home doesn't need the full BTU load, efficiency may be higher, something like 2.6 COP.

If we assume your oil boiler has poor efficiency, let's guess and say 65%, oil has a BTU of 90,000 adjusted for efficiency. divided by 3412 btu per kWh of electricity, divided by the COP (we're guessing at the 2.6 since we don't know the load or the efficiency of the gree at a lower capacity), times your electric rate of $.482, we get $4.89. Oil would have to be more than $4.89 per gallon to make you want to use your heat pump all year. Certainly below 27°, you'd want to use oil. Now, if your heat pump needs to pump out it's full capacity at 27°, the COP is 1.76, using that same formula above, Oil would have to be more than $7.22 a gallon for you to want to use electricity. So certainly, you'd want to use oil if your heat pump is operating at high speeds.

I should make note that when I say "you'd want to use oil" I mean from a financial standpoint - I always get one or two comments about environmental impact. And of course, I am only guessing at your oil boiler's efficiency, good ones are in the high 80's, you mentioned an old one, so I went with 65%, Same with the COP assuming your home doesn't need it's full 24,000 btu heating load, if it does, you'll be closer to that higher break even point for oil, meaning you'll want to be using oil, certainly with temperatures below around 30°. Check with gree and your specific unit to see if they have better numbers for you, and if you can get an energy meter on your unit to see how many kw your unit pulls to see how close to capacity it is running at any given point.