My two cents. The moment AI is coding itself, it stops being controlled. GPT-4 is already testing better than the majority of humans on an incredibly broad range of subjects. In some number of iterations, 2 or 50, I don't know, it will be smarter and more capable than any group of humans. Nobody's controlling that and power structures will change accordingly. The belief that AIs will decide to be our Nannies and just take care of us seems optimistic. Ambivalence towards us from an advanced AI seems likely. God forbid it decides we are a hindrance to its goals.

But, it will be built off datasets we feed it. Carefully curating those datasets will be biased, but is also our best bet, IMO.