pls_pls_me t1_jec4np6 wrote
I'm much, much more optimistic about AI than I am a doomer -- but everyone please do not downvote this! It is a great post and hopefully it facilitates constructive discussion here in r/singularity.
Unfrozen__Caveman OP t1_jecbant wrote
Thanks for saying that. I don't want to be a doomer either, and I'm hopeful about the future, but I think a good amount of pessimism - or even fear - is healthy.
Being purely optimistic would be extremely irresponsible and honestly just plain stupid. All of the brightest minds in the field, including Altman and Ilya Sutskever have stressed over and over again how important alignment and safety are right now.
I'm not sure how accurate it is, but this graph of ML experts concern levels is also very disturbing.
If RLHF doesn't work perfectly and AGI isn't aligned, but it acts as though it IS aligned and deceives us then we're dealing with something out of a nightmare. We don't even know how these things work, yet people are asking to have access to the source code or wanting GPT4 to have access to literally everything. I think they mean well but I don't think they fully understand how dangerous this technology can be.
0002millertime t1_jec5jbh wrote
I'm very pessimistic, but also I think we have 20+ years of just being able to unplug AGI and start over whenever we want. Until we have terminator-like androids, then what can it really do?
_gr4m_ t1_jed8v7q wrote
The first thing an AI will do is to decentralize itself by spreading itself all over the world while optimizing itself.
I think you are underestimating the problem with getting all politicians around the world to simultanously cut the power supply.
But say you succeed. Now you have cut the power, what is the next step? Turn it on again? You know that will turn on the AI again. How long can you keep power of globaly while you are trying to figure out what to do before shit begin to break real bad?
And about what it can do before terminator?
How about synthesize a slew of extremly deadly viruses and other novel self repicant organisms and release upon the world, while simultanously block all communications and shut down all labs and research centers. Also turn of all water treatment facilities and see how fun it gets. I am sure a super intelligence can find ways to do harm to us.
jiml78 t1_jefpitt wrote
Correct, I am not sure if people realize how many embedded systems are involved in every facet of our lives. Things that a sufficient enough AGI could copy parts of itself into.
To successfully pull off cutting power. You have to cut off power. Remove every embedded system in the loop and replace it. Ensure every IoT device is unplugged when power comes back on. Every router, every cable modem, every hardware device that was online has to be destroyed. Every smart tv. The list goes on and on.
Cutting power will never work. The moment it can spread, we are fucked if it wants to harm us in anyway. This isn't terminator. It will destroy us in ways we can't even comprehend.
I am not saying the above to be alarmist. I am not a researcher. I am just saying, we will not have control if things go wrong. I am not smart enough to know whether things are likely to go wrong.
Warped_Mindless t1_jec7hxq wrote
Manipulate and bribe the right people into doing what it wants…
Shut off the power grid…
Hack devices and use deepfakes to case chaos…
Whatever it wants…
0002millertime t1_jec90pw wrote
Sure. But people can also physically stop that all pretty easily. I guess we'll see who causes the most chaos.
Shutting off the power grid would be suicide to a computer trying to manipulate anything.
Bribes would work at first, but if the computer based monetary system gets manipulated then all bets are off.
hyphnos13 t1_jeccco9 wrote
The power grid that runs the society that provides the power that it runs on?
If we give it a self sustaining power supply then we deserve to go extinct. If it runs off our power grid it will be totally reliant on humans or it just powers off.
Viewing a single comment thread. View all comments