“Malice” is something we ascribe to a person with intent. An AI is not capable of intent, which is why it’s not capable of malice. But that also means it cannot exist independently than humans. It will always be a tool humans make and humans evaluate. So, you’re still going to be choosing between humans, not an AI against a human.

And unfortunately, though the AI cannot have malice, it can fail successfully. Consider giving the AI a directive “minimize long-term human suffering.” It may determine that killing everyone instantly is the best way to guarantee that. Qualifying that reward policy is harder than you think.