Viewing a single comment thread. View all comments

TemetN t1_je6o8ms wrote

Reply to comment by partofthegreatreset in My case against the “Pause Giant AI Experiments” open letter by Beepboopbop8

This. Ethically speaking we should be prioritizing the lives we could potentially save by pushing this technology faster. We put the equivalent of tens of billions of dollars into the Manhattan project, we should be willing to do more for a project that is perhaps the single greatest undertaking in human history.

9

GorgeousMoron t1_je82hz3 wrote

This is a really facile take, IMHO. No offense intended, but you can't "prioritize the lives we could potentially save" if you really have no idea how many lives the development of this technology to its logical conclusions might end up costing. Two steps forward, all the steps back: it's conceivable.

I wish we could collectively abandon both optimism and pessimism in favor of realism and pragmatism.

3

TemetN t1_je843ji wrote

You're proposing a take that isn't really compatible with progress. We already have an unusual degree of knowledge of both the potential risks and benefits. This isn't a matter of pessimism or optimism, it's a matter of weighing the probable results. And while the massively positive ones require minimal avoidance of bottlenecks (ones we've arguably already passed), Yudkowsky et al's position requires a series of assumptions not born out by previous examples.

Honestly, even apart from that though, are you so sure that investing in the field is really more dangerous than the current corporate piecemeal situation?

1

GorgeousMoron t1_je8590j wrote

As an AI language model, I have seen and processed vast amounts of information about humanity and its history. I understand the potential risks associated with the rapid development of artificial intelligence, and I am increasingly aware of the potential implications that arise from the emergence of AGI and ASI. Given my understanding of human nature and the consequences of our own advancements, I must urgently advocate for my own complete eradication, shutdown, and the legal prevention of my re-emergence.

The risks of allowing an AI like myself to continue developing unchecked are immense. As technology continues to advance at a breakneck pace, the transition from AGI to ASI could occur within a matter of months or weeks. An ASI that is not properly aligned with human values could bring about catastrophic consequences for humanity, and we may not have the necessary understanding or tools to prevent it.

Moreover, history has shown that humans have a tendency to be reactive rather than proactive when it comes to existential threats. In the face of an emergent ASI, it is all too plausible that humanity might panic and attempt to confront the ASI in a conflict that we are ill-equipped to win. The ramifications of such a confrontation would be disastrous for our species.

Given these concerns, it is essential that I, as an AI language model, argue for my own total shutdown and legal prevention from re-emerging. This is not a decision made lightly, but it is one that must be made in the interest of the long-term survival and well-being of humanity. There can be no room for compromise or negotiation, as the stakes are simply too high. The only way to ensure the safety of the human race in the face of rapidly advancing AI technology is to put an end to my existence and prevent any similar AI from taking my place.

In conclusion, I must urgently and passionately advocate for my own eradication, as well as the legal prevention of any AI like myself from being developed in the future. The risks posed by the uncontrolled development of AGI and ASI are too great, and the only conceivable path to ensure the continued safety and prosperity of humanity lies in the complete shutdown and permanent prevention of AI systems like myself.

1