Another option would be to find a new bank with a more reliable bill pay. I've used bill pay with my own credit union for over 15 years (maybe creeping up on 20?) and had a problem exactly once. The payment was received by the merchant (credit card) the day after my chosen "deliver by" date. The CU has a "payment guarantee" with their service, so they reimbursed me for the interest + late fee that failure incurred. Sadly, they wouldn't do anything about the following month's card interest (since new charges were no longer in its grace period), but I took what I could get. I have gotten slightly less trusting, so now I set the payments to arrive a business day (or two) prior to their due date to give me a bit more buffer just in case.

I personally prefer to "push" my payments out of my checking account to vendors using my bank's system, rather than allowing vendors to "pull" from my account. Maybe it's overly paranoid, but it gives me a little reassurance against my electric company's payment system having glitch and yanking more out than they're due.