Viewing a single comment thread. View all comments

impossiblefork t1_j9sacbf wrote

I think in the opposite way: if alignment is possible, then alignment is profoundly dangerous.

If alignment is possible, then the AI can be aligned with the interest of the wealthy capital owners who fund its development, and can be used to basically control the world.

Meanwhile, if alignment is impossible, ordinary people who have access to these hypothetical future 'superintelligences' can convince these entities to do things that they like, but which are undesired by the model-owning class.

For this reason, if we are on some kind of path to super AI, the development of technology to permit value alignment must be prevented.

2

sam__izdat t1_j9sj2zl wrote

> I think in the opposite way: if alignment is possible, then alignment is profoundly dangerous.

Exactly. What is this neoliberal fever dream? "But what if the computer doesn't do what they want?!" -- my god, what if it does? Are we living on the same planet? Have you seen what they want?

I love how the core of the panic is basically:

"Oh my god, what if some kind of machine emerged, misaligned with human interests and totally committed to extracting what it wants from the material world, no matter the cost, seeing human life and dignity as an obstruction to its function?!"

Yeah, wow... what if?! That'd be so crazy! Glad we don't have anything like that.

7

okokoko t1_j9srgl5 wrote

>Meanwhile, if alignment is impossible, ordinary people who have access to these hypothetical future 'superintelligences' can convince these entities to do things that they like

Interesting, how are you gonna "convince" an unaligned AI though, I wonder. I feel like there is a flaw in your reasoning here

5

impossiblefork t1_ja6rt6s wrote

I doubt it's possible, but I imagine something like [ed:the] DAN thing with ChatGPT.

Most likely you'd talk to the AI such that the rationality it has obtained from its training data make it reason things out that it's owner would rather it stay silent about it.

1