I think the challenge will be finding examples to "program" the model with. Remember, these AI models aren't programmed with rules, they are shown millions of examples of interactions and trained to recognize good ones.

It's very much like training a puppy, you can't just tell it "don't chew on my stuff". You need to keep practicing similar situations over and over and rewarding only the desired behavior. In the end, your puppy usually doesn't chew on your stuff, but you don't know exactly what it's thinking.

The new chat AIs take that model trained on good conversations and essentially keep trying out responses internally until they find one that matches the good detector.

The challenge with your idea would be gathering millions of examples of discourse effective at changing people's minds.