>But the thing is we just don't know when AGI is coming, maybe a long way off.

This is what I feel a lot of people don't get. We have literally no idea what the threshold for consciousness is. We don't even know how to identify it in other humans let alone another species. Without knowing what that threshold is, there is absolutely no way for us to determine how close or far away from it we are. All we do know is that if and when AI reaches that level, it will intellectually outpace humans at a significant rate.

When the first atomic bomb was created, scientists knew precisely under what conditions a nuclear reaction would go critical. Now imagine if those scientists had absolutely no way of know when or if the reaction would go critical and blow up in their face. That is exactly what we are doing with AI. Racing towards a criticality point which we cannot identify.

Long story short, it could happen in 10 years or 100 years. We literally have no means knowing when.