Sigura83 t1_iybiukl wrote on November 30, 2022 at 4:04 AM

Reply to comment by FirstOrderCat in Better Language Models Without Massive Compute by Tom_Lilja

I was impressed by the Google Say-Can project. I think we'll see an AI model that can do science pretty soon, it's the goal of Deepmind, and they've done amazing things. And, I think Google moved to a Transformer AI for search two years ago...

I did read a recent paper saying LLM were just doing fancy statistics and hardly thinking at all, but if it's sorta doing it now in only a few years it'll surpass Human level.

If I were to bet, I would say an evolutionary algo + interacting LLM based robots will be beyond anything we can imagine. First in a MMO like setting, then the real world. But we'd need super duper computers to do it, which we might not get because Moore's law may be done. Well... hmm... if the AI MMO was text based, the LLMs could run wild and evolve away without costing too much energy

I dunno... just dreaming! ^_^;;;

FirstOrderCat t1_iybm4dy wrote on November 30, 2022 at 4:33 AM

>First in a MMO like setting, then the real world.

nop, this transition is very hard because of following reason:

current wave of AI can approximate giant datasets, that's something it is doing very well. So, all your examples is: they throw terrabites of data on neural network, and it learns patterns. But this kind AI can't generalize and do abstract thinking which means it can't learn from very few examples.

Meaning yes, they can ask AI to play MMO 100 millions times and it will learn from its own mistakes, but you would need to do the same 100 million times in real world, which is not very feasible.

Another issue is that: MMO has much smaller level of freedom than real world, which makes MMO is not a good benchmark.