The part that really made the article worth reading was this:

> Like ChatGPT, PaLM + RLHF is essentially a statistical tool to predict words. When fed an enormous number of examples from training data — e.g., posts from Reddit, news articles and e-books — PaLM + RLHF learns how likely words are to occur based on patterns like the semantic context of surrounding text.

So, even when you ask it to create a completely new fictional mishmash story about Darth Vader landing his Death Star in Aragorn to save Thor from being assimilated by the Borg, it will spew out sensible sounding sentences because it knows those references and what comes before and after those words (e.g. Darth Vader, Aragorn, Thor, Borg) and how to link the "before" and "after" words to stitch up a story by combining the same / common "before" and "after" words of the others.

It gives an impression of really understanding what it is saying in some sense, possessing mental models of some sort. But it does not. And that is why it will at most be the next replacement of web search - the truly smart assistant.

But it is nowhere close to real intelligence of any kind because it has no model of reality.

It is great and useful and will it make money and result in productivity and economy? Absolutely, it will change computing services dramatically.

Is it intelligence? Nope. Not even close.

As long as AI continues to be trained on data from the internet, "average plus epsilon" is the best we can hope for.