Viewing a single comment thread. View all comments

Frumpagumpus t1_j8f2s71 wrote

neither altman nor yudkowsky are whiz bang programmers or computer scientists

academic computer science basically ignores the concept of the singularity as not relevant to their more specific research goals.

amongst rationalists, maybe more are sympathetic to yud/bostrom because he kind of founded the movement, and they are interested in managing existential risk and have a kind of technocrat neolib/socialist top down planning bias just due to the demographic composition of the community

amongst venture capitalists, obviously altman is more respected

i lean team altman, although I don't think the primary denizens of future society will be humans lol. Also I don't think it will be complete utopia but definitely way cooler than our society is. More vitality/thought/energy, less of a doomer/malthusian vibe

I would say let's ask instead what vernor vinge or von neumann thinks XD

(also venture capitalists basically = tech founders so they are less armchair quarterbacks, and typically have ivory tower credentials but also ground floor experience)

17

Unfocusedbrain t1_j8fcuq2 wrote

> Also I don't think it will be complete utopia but definitely way cooler than our society is. More vitality/thought/energy, less of a doomer/malthusian vibe

I believe the same. After a certain point the sole-currency becomes energy, space, and matter. There isn’t an infinite amount of it, but for human purposes there are and so it will feel like complete utopia/communist paradise. If AI can build anything with enough matter and energy, and can allow any place to be habitable, well that eliminates currency except for really extreme scenarios.

I think at macro level there will be questions of “Who pays the cosmic water, energy bill and rent?” At that point it would be in the hands of AI systems so far advanced that they can manage those concerns without issue.

11

Frumpagumpus t1_j8ffc5p wrote

assuming there is a future I think there will still be something analogous to currency that facilitates trade, though our currency essentially is a scalar and it's possible future currency will be a matrix or a vector (e.g. add some extra values to represent externalities or something). maybe essentials of energy/space/matter would be extremely cheap though, although it's possible with massive computational speedup in thought there could also be an increase in consumption of some combination of those as wells by whatever agents inhabit the society. idk really hard to say but I'm betting on a dyson swarm of some kind lol (hard to imagine what that much energy could be used for other than like super powerful simulations though). Can also imagine literal mind viruses or some scary shit like that.

6

sticky_symbols t1_j8g5wkl wrote

Good sociopolitical breakdown.

But biases aren't the whole story - there's a lot of logic in play. And way more of it is deployed on one side than the other...

4

Frumpagumpus t1_j8gjgcr wrote

personally i am not sure how useful logical reasoning is in exploring the "phase space" of super intelligence. my intuition would be anything short of a super intelligence would be pretty bad at sampling from that space.

i do think something like computational complexity theory could say a few things, but probably not too much that is interesting or specific

like with a kid parents set initial conditions but environment and genes tend to overrule them eventually

2

sticky_symbols t1_j8iu8wz wrote

Yeah. But if we get it wrong, we're all dead. So we have to try.

2