Viewing a single comment thread. View all comments

michael_mullet t1_ix0nvq2 wrote

Reply to comment by TemetN in 2023 predictions by ryusan8989

>Gato 2 (or whatever they call the scaled Gato they're working on) drops, confirms scale is all we need.

If scale is all we need, AGI by end of 2023.

It may not be released publicly but will become apparent to those in the industry and copied where possible.

I am not convinced that scale is all we need but would be happy to be wrong.

18

TemetN t1_ix0okxi wrote

I'm (repeatedly) on record as expecting AGI (as in the Metaculus weak operationalization) by 2025. So while I broadly agree with this, I do think it only applies to a relatively specific and closer to the original use of the term, rather than the more volitional use.

11

michael_mullet t1_ix18qed wrote

I think I understand you. Likely scale is all that is needed for a non-volitional AGI, and that may be all that is needed for accelerating technological change. Humans can provide the volitional aspect of the system.

7

-ZeroRelevance- t1_ix1crev wrote

Do we even want a volitional AGI though? A non-volitional AGI seems like all the benefits with none of the problems. Since the main draw of an AGI is the problem-solving aspects, which you don’t need volition for.

Also, it shouldn’t have any problems pretending to be one if we want it to though, given how current language models already make very convincing chatbots. It’s just that in such a case, we’d ultimately stay in control, since a non-volitional AI would have no actual desires for things like self-preservation

9

TemetN t1_ix1fdw6 wrote

This. Plus I think that volition is unlikely to be simply emergent, which means that it's likely to take its own research. And I don't see a lot of call for, or effort at researching in such a direction (Numenta? Mostly Numenta).

5

CosmicVo t1_ix2q8f1 wrote

Scale is indeed not all we need. In fact GPT-4 has less parameters than GPT-3. Or the same. Idk. Anyway the focus is shifting toward trainingdata (e.g. learning rate, batch size, sequence length, etc). They’re trying to find optimal models instead of just bigger ones. Hyperparameter tuning is unfeasible for larger models but result in a performance increase equivalent to doubling the number of parameters.

4