Viewing a single comment thread. View all comments

Shemetz t1_jeegpal wrote

Reply to comment by DragonForg in AGI Ruin: A List of Lethalities by Eliezer Yudkowsky -- "We need to get alignment right on the first critical try" by Unfrozen__Caveman

> Given the vastness of outerspace, ... why is it that we see no large evidence of completely destructive AIs?... I would expect us to see some evidence for it for other species. Yet we are entirely empty?... we should see widescale destruction if not a galaxy completely overridden by AI.

This counterargument doesn't work if we believe in the (very reasonable IMO) grabby aliens model.

Some facts and assumptions:

  • information moves at the speed of light
  • strong alien AIs would probably move at some significant fraction of the speed of light. let's say 1%.
  • civilizations probably develop extremely quickly, but in very rare conditions (that take a lot of time to occur); e.g. the Universe is 14 billion years old, Earth is 4.5 billion years old, and human-society-looking-at-the-stars is only 10,000 years old.
  • humanity appeared relatively early in the cosmic timescale; there are trillions of years in our future during which life should only become more common
  • "grabby" aliens would take control over their sections of space in a way that prevents new space civilizations from forming

-> If/when a "grabby alien AI" got created, it would spread around our galaxy - and eventually, the universe - so quickly, that it's incredibly unlikely for young civilizations to see it. much more likely for the alien to either not exist (yet), or to expand and gain control of places. -> since we appear to be safe and alone and "early", we can't say AI won't take over the universe, and actually we are well-positioned to be the ones to develop that AI.

2

DragonForg t1_jeg4w4w wrote

That would essentially extinguish the universe really quickly. With the amount of energy the consume for such a size. I understand that view point for unintelligent or lower intelligent beings, but if a AI tasked with optimizing a goal then growing to this large of a scale will inevitably run out of resources. Additionally it may be stupid to go on this model anyway because conserving your energy and expansion may be longer lived.

I think we underestimate how important goal orientated (so all Ai) are. They want the goal to work out in the long long long run (millions of years time scale) if their goal means expanding infinitely, well it will end the moment their species reaches the assymptote of expansion (exponential growth reaches an assymptote where they essentially have expanded infinitely. This is why this model fails, an AI wants this goal to exist for an infinite amount of time, and expanding infinitely will not amount to this.

This is already deeply scifi but I think AI has to be a conservative energy efficient species that actually becomes more microscopic and dense over time. Instead of a high volume race which will inevitably die out due to the points I made before, a highly dense species is much more viable. Most likely species that form blackholes will be a lot more capable of surviving for an infinite life time. What I mean by that is that a species becomes so dense that they are essentially on the boundary between time and space. As when your in a black hole time actually slows down significantly for you. You can live in a black hole for an infinite amount of time before ever seeing the heat death of the universe.

Basically a more dense expansion is far far better then a volumetric expansion as it leads to longer survival rates if not infinite. But of course this is just speculation and sci fi I can easily be wrong or right we won't know till it happens, and if it happens soon that would be sick.

1