This isn't a negotiation. You don't need his permission to visit your parents and if he feels like he's not 'adult' enough to be left on his own for a weekend, he's got bigger problems to work on. No matter when you go, if he doesn't go HE WILL BE ALONE. He does not have a valid argument.

Him telling you he regrets marrying you? Looks like you might want to extend your time at your parent's place so you can take time to consider whether you want to stay with someone this purposefully hurtful and nasty.