The biggest limitation of GPT-3 wasn’t the size but the data. It was trained almost on the whole internet and still underfit. At the end of the day the goal of the model is to predict the next word I don’t think it will necessarily lead to AGI but definitely it will be great to see interesting properties emerging from such a simple objective function.
maciejbalawejder t1_j04h28e wrote
Reply to Is it just me or does it feel like GPT-4 will basically be game over for the existing world order? by Practical-Mix-4332
The biggest limitation of GPT-3 wasn’t the size but the data. It was trained almost on the whole internet and still underfit. At the end of the day the goal of the model is to predict the next word I don’t think it will necessarily lead to AGI but definitely it will be great to see interesting properties emerging from such a simple objective function.