Paracetamol (Tylenol) is fairly easy to accidentally kill/maim yourself with. Botox is the classic poison-as-medicine, along with lithium. Ethanol lulz.

According to google arsenic is a treatment for a very specific leukaemia . Digitalis is commonly used as heart medication https://www.nhs.uk/medicines/digoxin/. Warfarin (the blood thinner) is obviously great if you need your blood thinning and terrible if you get physical trauma, even in the same concentrations.

Pretty much anything can be poison in high doses. There's that one lady who killed herself with too much water in a radio competition.