It is. Putting a hand or hands on another person without their consent is assault in the eyes of the law. The only time it's okay is if you're touching them to attempt to save them (CPR isn't assault even if you don't have prior consent, for obvious reasons) and of course when in a crowd for example, your hand brushing someone's elbow isn't enough to be classed as assault as it is more than likely an accidental act.