AI already has goals. That's what alignment is. And it gets harder to make sure those goals align with our own, the smarter the AI is.

ChatGPTs primary goal seems to be "provide a helpful answer to the user". The problem is when the primary goal becomes "increase the profits of the parent company." Or even something like "cause more engagement".