Idk. Seems like you did this to yourself by opening the relationship up to other people. He slept with another woman just like you said he could and you didnt like that and the entire thing ended after you both agreed. It wasnt as hidden or seperate from him as you thought it was id imagine... it really seems like you both did it to eachother and it's still effecting you both.

This is something you shpuld both be talking to eachother about honestly. Rather than an internet forum