It's the plastic heating up and melting while the bit is going through. Might sound stupid but chill the bit and the box before drilling and that should help, also try to go through faster, but stay safe. You don't want a drill bit in your hand or thigh.

And yes that bit is going to be subpar for the work you're trying to do. Harbor freight has cheap bits that'll do what you want https://www.harborfreight.com/power-tools/drill-driver-bits/high-speed-steel-drill-bit-set-13-piece-61723.html

These ones won't be good for much, but the light plastic, they'll be great.