Persuasion is the one thing chatgpt can do really well. That's something that doesn't require any hard logic. And it's also why this tech is dangerously deceptive, it will be so persuasively correct until it's not.

Eventually AI will be able to build a better legal defense than a human can, and in that case it would be unethical to give people human defense teams.

However, that day is not today. ChatGPT has no hard internal logic. You can trick it into doing bad math for example, or sometimes it writes code that is just wrong.

I'm no lawyer but I'm assuming legal defense requires some sort of presentation of factual evidence, logic, and verification of that evidence. Right now you can't guarantee the AI hasn't just spat out a huge document of gibberish that looks right but has a hidden logical flaw.

This whole thing is just dancing around the true issue, which is why not make actual privacy laws? Like Google/Amazon/Facebook doing the sideeye monkey meme right now.