Practical-Mix-4332 OP t1_j03ujzs wrote on December 13, 2022 at 9:47 PM

Reply to comment by ChronoPsyche in Is it just me or does it feel like GPT-4 will basically be game over for the existing world order? by Practical-Mix-4332

I mean you can kind of extrapolate based on the difference between GPT 2 and 3, but yes you are correct it is all speculation.

ChronoPsyche t1_j03vrmh wrote on December 13, 2022 at 9:54 PM

No you can't extrapolate. There are reasons behind things. GPT3 and GPT2 are both transformer models. GPT4 will likely be a transformer model too. At best it will just be a better transformer model, but it will still have context window limitations that prevent it from becoming anything that can be considered "game over for the existing world order". It will likely just be a better GPT3, not AGI or anything insane like that.

manOnPavementWaving t1_j04bxs9 wrote on December 13, 2022 at 11:44 PM

I agree that you can't extrapolate, but it's definitely not the case that GPT4 has to have the same limitations as GPT2 and GPT3. Context window issues can be resolved in a myriad of ways (my current fav being this one and retrieval based methods could solve most of the factuality issues (and are very effective and cheap models, as proven by RETRO).

So I want to re-emphasize that we have no clue how good it will be. It could very well smash previous barriers, but it could also be rather disappointing and very much alike ChatGPT. We just don't know.

Practical-Mix-4332 OP t1_j0401zd wrote on December 13, 2022 at 10:23 PM

I don’t think it needs to be an AGI to make a huge difference though. If it really is much more impressive than GPT-3 it’s going to start causing massive shockwaves throughout society. It will bring AI to the public consciousness even more than it already is and make people start planning for that future instead of just imagining it as a hypothetical distant time.