If someone is asking if they are being annoying, it's because either A) they don't know because they don't understand how they are perceived by others but THEY WANT TO. B) They know they are annoying and are looking for a pass to keep going.

Either way, just say 'to be honest, yes' and take it from there. In the long run, it's the nicest thing you can do for A, and B hasn't earned the right to have their feelings protected.