DigThatData t1_j9rux16 wrote
I think the whole "paperclip" metaphor descibres problems that are already here. a lot of "alignment" discussion feels to me like passengers on a ship theorizing what would happen if the ship became sentient and turned evil and decided to crash into the rocks, but all the while the ship has already crashed into the rocks and is taking on water. It doesn't matter if the ship turns evil in the future: it's already taking us down, whether it crashed into the rocks on purpose or not. See also: contribution of social media recommendation systems to self-destructive human behaviors including political radicalization, stochastic terrorism, xenophobia, fascism, and secessionism. Oh yeah, also we're arguing over the safety of vaccines during an epidemic and still ignoring global warming, but for some reason public health and environmental hazards don't count as "x-risks".
royalemate357 t1_j9ryzg7 wrote
>I think the whole "paperclip" metaphor descibres problems that are already here
Does it? My understanding of the paperclip metaphor is that an advanced AI will pursue its own goals that are totally unrelated to human goals, e.g. creating as many paperclips as possible. But AIs aren't advanced enough right now to be at this point.
As for what constitutes "x-risks", AFAIK it means "existential risk" which is like all of humanity going extinct. IMO the reason why people consider advanced AGIs an x-risk, and the others are not, is because the other problems you mentioned don't result in the extinction of *every* single human on Earth
DigThatData t1_j9rzrzd wrote
if a "sufficiently advanced AI" could achieve "its own goals" that included "humanity going extinct" (at least as a side effect) in such a fashion that humanity did the work of putting itself out of extinction on its own needing only the AGIs encouragement, it would. In other words, the issues I described are indistinguishable from the kinds of bedlam we could reasonably expect an "x-risk AGI" to impose upon us. ipso facto, if part of the alignment discussion is avoiding defining precisely what "AGI" even means and focusing only on potential risk scenarios, the situation we are currently in is one in which it is unclear that a hazardous-to-human-existence AGI doesn't already exist and is already driving us towards our own extinction.
instead of "maximizing paperclips," "it" is just trying to maximize engagement and click-through rate. and just like the paperclips thing, "it" is burning the world down trying to maximize the only metrics it cares about. "it" just isn't a specific agent, it's a broader system that includes a variety of interacting algorithms and platforms forming a kind of ecosystem of meta-organisms. but the nature of the ecosystem doesn't matter for the paperclip maximization parable to apply.
royalemate357 t1_j9s125d wrote
> instead of "maximizing paperclips," "it" is just trying to maximize engagement and click-through rate. and just like the paperclips thing, "it" is burning the world down trying to maximize the only metrics it cares about
Isn't there a difference between the two, because the latter concerns a human trying to pursue a certain goal (maximize user engagement), and giving the AI that goal. and so arguably, the latter is "aligned" (for some sense of the word) to the human that's using it to maximize their engagement, in that its doing what a specific human intends it to do. Whereas the paperclip scenario is more like, human tells AI to maximize engagement, yet the AI has a different goal and chooses to pursue that instead.
DigThatData t1_j9s23ds wrote
> Isn't there a difference between the two, because the latter concerns a human trying to pursue a certain goal (maximize user engagement), and giving the AI that goal.
in the paperclip maximization parable, "maximize paperclips" is a directive assigned to an AGI owned by a paperclip manufacturer, which consequently concludes that things like "destabilize currency to make paperclip materials cheaper" and "convert resources necessary for human life to exist into paperclip factories" are good ideas. so no, maximizing engagement at the cost of the stability of human civilization is not "aligned" in exactly the same way maximizing paperclip production isn't aligned.
royalemate357 t1_j9s2pf3 wrote
hmm i didn't realize that the origin of the paperclip maximizer analogy, but it seems like you're right that some human had to tell it to make paperclips in the first place.
DigThatData t1_j9s4kj0 wrote
ghostfuckbuddy t1_j9t2fqk wrote
Corporations are paperclip maximizers, sometimes literally.
Viewing a single comment thread. View all comments