Here's Ilya Sutskkever during a conversation with Jensen Huang on LLM being a simple statistical correlation.
>The way to think about it is that when we train a large neural network to accurately predict the next word in lots of different texts from the internet, what we are doing is that we are learning a world model.
>
>It may look on the surface that we are just learning statistical correlations in text, but it turns out that to just learn the statistical correlations in text, to compress them really well, what the neural network learns is some representation of the process that produced the text.
>
>This text is actually a projection of the world. There is a world out there, and it has a projection on this text, and so what the neural network is learning is more and more aspects of the world, of people, of the human conditions, their their their hopes and dreams, and their interactions and the situations that we are in, and the neural learns a compressed abstract usable representation of that. This is what's being learned from accurately predicting the next word.
>
>And furthermore, the more accurate you are in predicting the next word, the higher fidelity, the more resolution you get in this process.
The chat is available to watch officially on the Nvidia site if you're registered for GTC. If not, there's an unofficial lower-quality YouTube upload as well.
Being too reductive is still technically correct, but there're understanding of emergent properties left unexplored as well. Mitochondria is a collection of atoms vs Mitochondria is the powerhouse of the cells.
SnooWalruses8636 t1_je8ap4s wrote
Reply to comment by StevenVincentOne in The argument that a computer can't really "understand" things is stupid and completely irrelevant. by hey__bert
Here's Ilya Sutskkever during a conversation with Jensen Huang on LLM being a simple statistical correlation.
>The way to think about it is that when we train a large neural network to accurately predict the next word in lots of different texts from the internet, what we are doing is that we are learning a world model.
>
>It may look on the surface that we are just learning statistical correlations in text, but it turns out that to just learn the statistical correlations in text, to compress them really well, what the neural network learns is some representation of the process that produced the text.
>
>This text is actually a projection of the world. There is a world out there, and it has a projection on this text, and so what the neural network is learning is more and more aspects of the world, of people, of the human conditions, their their their hopes and dreams, and their interactions and the situations that we are in, and the neural learns a compressed abstract usable representation of that. This is what's being learned from accurately predicting the next word.
>
>And furthermore, the more accurate you are in predicting the next word, the higher fidelity, the more resolution you get in this process.
The chat is available to watch officially on the Nvidia site if you're registered for GTC. If not, there's an unofficial lower-quality YouTube upload as well.
Being too reductive is still technically correct, but there're understanding of emergent properties left unexplored as well. Mitochondria is a collection of atoms vs Mitochondria is the powerhouse of the cells.