currentscurrents OP t1_j62auto wrote
Reply to comment by rjromero in [R] Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers by currentscurrents
Yes, but I don't want to create too much optimism; meta-learning was also a promising lead when Schmidhuber wrote his PhD thesis.
Honestly, I'm not sure much has changed since then other than we got more compute power. Transformers are reportedly equivalent to 1990s meta-learning networks except that they run better on GPUs, and GPUs have gotten powerful enough to run them at very large scale.
lookatmetype t1_j62j0t3 wrote
is there anything he hasn't done?
Acceptable-Cress-374 t1_j62ql8t wrote
Stable diffusion with proper hands? :)
Viewing a single comment thread. View all comments