Viewing a single comment thread. View all comments

EddgeLord666 t1_j7lh7nj wrote

Reply to comment by drekmonger in 200k!!!!!! by Key_Asparagus_919

Well unlike most people on this sub, I think transhumanism should be prioritized over the creation of AGI. I’m more interested in AI serving us as tools or augmenting our capabilities than ruling over us. Furthermore, you absolutely could have continuity of consciousness as long as augmentation happened in a ship of Theseus way, say by gradually boosting your IQ by 20 points every year instead of all at once.

1

drekmonger t1_j7lia04 wrote

The Singularity, as it was originally imagined, included potential scenarios for transhumanism over a technological singularity. The original essay is still well worth the read, even 30 years later.

But the doomsday scenario the essay was ultimately warning against was that the Singularity would occur rapidly as a shocking cascade of events.

Perhaps in the "pet human" scenario, a benevolent ASI might slowly augment people as individuals.

Regardless, the problem is one of alignment, and I don't think you or I have much say in that. Even if a relatively benevolent organization like OpenAI develops the first AGI, their competitors (like, say, China's AI research efforts) won't be so benevolent.

As in capitalism, the most unethical strategy will tend to dominate ethical strategies. The "bad" AIs will win any race.

3

EddgeLord666 t1_j7livt6 wrote

So far we are not at the stage where the Singularity is likely to be imminent, contrary to what some people here say. That means we probably have anywhere from 1 to 3 decades for the “good” people to coordinate and plan ways for it to happen in a more beneficial way or stop it from happening at all if that is deemed more desirable. That is really what people should be using this sub for, not just idle speculation.

2