In theory, reinforcement learning could solve this, but specifying the reward function is very hard and if you need human evaluation then it is very slow. I think it’s not quite as theoretical of a solution as evolutionary algorithms, as it’s been successfully applied to create very capable game playing agents before, but I think it’s very difficult to beat behavior trees or simple utility systems when it comes to creating characters that are fun to play with.

Yes. Exponential functions look like s-curves when applied to reality.

By definition, most people will be mediocre

Bit of a nitpick but Transformers were invented by Google Brain, not DeepMind.

Historically better abstractions have not lead to less developer demand, quite the opposite. It’s what programmers have been doing since the invention of programming.

The only reason we think other people are conscious is via analogy to ourselves. And the only reason we think animals are conscious is via analogy to humans. The truth is we really have no idea what consciousness is nor do we have a way to measure it, so any ethics based on a rigid definition of consciousness is very fallible.

Loved the first half of the first season, didn’t like the second half.

How would a world look like where we have AGI but it still takes another 10-20 years to reach LEV?

People in wealthy nations already put off having kids until later than their parents and grandparents. As people live longer this trend will most likely continue. Why not wait until your 50s (or even longer) to have kids if by then you’re still biologically 30? I think it’s much more likely that population will stagnate or decline as a result of curing aging.