I think you need to establish what you both want out of this relationship. Sounds like he may only be interested in sex and thought that's what you wanted too so now he feels the rug has been pulled from under him.

But at no point due you owe him sex or your body.