What mic are you using? If it's the modmic then I guess it's the terrible mute slider of it, it's basically broken.

Can recommend to simply set a hotkey for muting yourself inside of whatever software you're using instead of using mute buttons. Worked for me and transitioning over to these settings just takes a bit to get used to.