No, you should be able to defend yourself against someone using violence against you.

It isn't about property or killing them. It's about protecting yourself from serious harm or death. If a 90 lb. school kid shoves you, you don't get to kill them, because a reasonable person wouldn't consider that to be a serious/deadly attack.