Viewing a single comment thread. View all comments

sam__izdat t1_j9si9wx wrote

Worry less about misalignment of Skynet, the impending singularity and the rise of the robots, which is science fiction, and worry more about misalignment of class interests and misalignment of power, which is our reality.

For the former, it's still mostly an ambitious long-term goal to simulate the world's simplest nematodes. There's hardly any reason to believe anyone's appreciably closer to AGI now than they were in the 1950s. For the latter, though, there are well-founded concerns that automation will be used for surveillance, disinformation, manipulation, class control, digital Taylorism and other horrifying purposes, as the species knowingly accelerates toward extinction by ignoring systemic failures like AGW and nuclear war, which pose actual, imminent and growing existential risks -- risks that will be compounded by giving state and capital tools to put the interests of power and short term ROI above even near-term human survival, let alone human dignity or potential.

"What if this pile of linear algebra does some asimov nonsense" is not a serious concern. The real concern is "what if it does exactly what was intended, and those intentions continue to see omnicide an acceptable side effect."

4