I don't know but it seems like LLMs will get there faster as soon as they become multimodal. Language is already symbol manipulation.

Previously from Deep Mind in the domain of symbolic reasoning:

Making Sense Of Sensory Input

>This paper attempts to answer a central question in unsupervised learning:what does it mean to "make sense" of a sensory sequence? In our formalization,making sense involves constructing a symbolic causal theory that both explains the sensory sequence and also satisfies a set of unity conditions. The unity conditions insist that the constituents of the causal theory -- objects,properties, and laws -- must be integrated into a coherent whole. On our account, making sense of sensory input is a type of program synthesis, but it is unsupervised program synthesis.
>
>Our second contribution is a computer implementation, the Apperception Engine, that was designed to satisfy the above requirements. Our system is able to produce interpretable human-readable causal theories from very small amounts of data, because of the strong inductive bias provided by the unity conditions.A causal theory produced by our system is able to predict future sensor readings, as well as retrodict earlier readings, and impute (fill in the blanks of) missing sensory readings, in any combination.
>
>We tested the engine in a diverse variety of domains, including cellular automata, rhythms and simple nursery tunes, multi-modal binding problems,occlusion tasks, and sequence induction intelligence tests. In each domain, we test our engine's ability to predict future sensor values, retrodict earlier sensor values, and impute missing sensory data.
>
>The engine performs well in all these domains, significantly out-performing neural net baselines. We note in particular that in the sequence induction intelligence tests, our system achieved human-level performance. This is notable because our system is not a bespoke system designed specifically to solve intelligence tests, but a general-purpose system that was designed to make sense of any sensory sequence.

Edit: Also check out the followup paper:

Making Sense Of Raw Input

Very cool! And this paper is from 2019-20, and some of those I listed in my post are from 2018-19. I wonder how many of these turned out dead ends, and how far did the rest go by now. Papers for major conferences are often preprinted in advance, but sometimes DeepMind also comes out with something like AlphaGo or AlphaFold on their own schedule. Maybe some highly advanced Gato 2.0 is just around the corner?

I think there's a fallacy here - it may be the case that language involves all manner of symbol manipulation in its extended manifestation, but there is still considerable evidence that LLMs are not wholly capable of what we mean when we talk about language. There are many capacities still missing in even the most powerful LLMs. It may be the case that more data and more scale and some clever tricks will resolve these issues—though I am skeptical—but from what I have seen, LLMs thus far demonstrate a very limited capacity for 'symbol manipulation.' Namely, they show capacities for generation of statistically plausible sequences of letters, but fail in obvious ways on other sophisticated forms of symbolic manipulation and reasoning.

I'd be curious to hear if you agree, or perhaps if you think that the current limitations in symbol manipulation will be overcome with more scale on same architectures? This was a core question in the AGI Debate hosted by Montreal AI last week, and it seems experts on the subject are quite divided.

Some thoughts and then some references to neuro-symbolic approaches:

The reality is that AGI has become (always been?) an engineering problem [For intellectual honesty: this is not consensus, e.g. among neuroscientists. See comments below]. Many times in the past we have seen less theoretically scalable methods outperform more principles ones, so nobody can predict which one will win in the short term. LLMs are promising since they can leverage all the hardware acceleration and the pre-existing work of different fields (NLP, Computer Vision, RL). So it may very well be that DL will be enough to achieve great results and more investment and optimisation will pile-in, making symbolic approaches comparatively less attractive to fund in the short term.

Who knows, maybe the right symbolic architecture has already been proposed 20-30 years ago and nobody took the effort to put into a modern GPU accelerated codebase.

https://arxiv.org/abs/2012.05876 Neurosymbolic AI: The 3rd Wave

https://arxiv.org/abs/2105.05330 Neuro-Symbolic Artificial Intelligence: Current Trends

https://arxiv.org/abs/2002.00388 A Survey on Knowledge Graphs: Representation, Acquisition and Applications

Language is syntax, and LLMs excel at it. I think it is interesting to note that GPT improved with learning programming because programming languages follows exact syntactic rules, which are rules of symbol manipulation. But it seems those rules are also great when applied to ordinary language which is much more fuzzy and ambiguous. transformers do seem to be exceptional at capturing syntactic relationships without necessarily knowing what it is that they are talking about (so, abstractly). And math is all about manipulating abstract entities..

I think symbol manipulation is something that transformers will continue to excel at. After all it's not that difficult either - Mathematica does it. The model may not understand the consequences of their inventions, but it will definitely be able to come up with proofs , models, theorems, physical laws etc. If the next GPT will be multi-modal, it seems it might be able to reason about its sensory inputs as well

> Who knows, maybe the right symbolic architecture has already been proposed 20-30 years ago and nobody took the effort to put into a modern GPU accelerated codebase.

I also half-expect that in ten years, what current LLMs do on racks of GPUs, will fit in a phone chip, because many advances in efficiency come from utilizing old simple techniques like Monte Carlo and nearest neighbors.

>Language is syntax

Language is much more than syntax; if language as pure syntax is your starting point then it's not really a conversation worth having IMO.

We never know. Imo LLM will grow exponentially. Today, they're data inefficient, but someday (definitely not tomorrow lol), more sentient than we can ever imagine.

Language is an action we take to achieve some short- or long-term intent by affecting others' actions. It just so happens that text data is (mostly) symbolic, so it appears like only a problem of symbol manipulation. The text that these models are trained on are observations of language production, where utterances are generated from intent (e.g., wanting to convince someone of some argument, wanting to sell something to someone) and context (e.g., what you know about your interlocutor). This doesn't even cover vocal / signed communication, which is much more continuous.

Intent and context are not purely symbolic. Sure, with infinite observations, that generative structure would be perfectly reconstructable. But we are nowhere near that, and humans are completely capable of modeling that generative process with very little data and continuous input (which we learn to discretize).

Just want to reiterate: if you think language is just syntax, I'd recommend listening to some linguistics lectures or reading a book or two on the subject (i.e.: books on language, not on syntax or grammar). John McWhorter has some very approachable and eye-opening Audible courses that might change your perspective.

I am aware of some of the philosophy of language, but i prefer to look at the neuroscientific findings instead. Language is a human construct that doesn't really exist in nature - communication does, which in humans is exchange of mental states between brains. The structure of language follows from abstracting the physical world into compact communicable units, and syntax is a very important byproduct of this process. I am more interested to see how hierarchical structure of language arises in these computational models like LLMs, which are open to empirical investigation. Most folk linguistic theories are high conjecture that has only circumstancial evidence.

Exactly, it's not just a matter of software architecture, but also of preexisting optimised libraries, hardware acceleration, economic incentives, funding. Very hard to predict how it will end up.

hey, I work on program synthesis, which is a form of neuro-symbolic reasoning. here's my take.

the word "neuro-symbolic" is thrown around a lot, so we need to first clarify which kinds of work we're talking about. broadly speaking there are 2 kinds.

1. neuro-symbolic systems where the symbolic system is _pre-established_ where the neuro network is tasked to construct symbols that can be interpreted in this preexisting system. program synthesis falls under this category. when you ask chatgpt/copilot to generate code, they'll generate python code, which is a) symbolic and b) can be interpreted readily in python
2. neuro-symbolic systems where the neural network is tasked to _invent the system_. take for instance the ARC task ( https://github.com/fchollet/ARC ), when humans do these tasks (it appears to be the case that) we first invent a set of symbolic rules appropriate for the task at hand, then apply these rules

I'm betting Demmis is interested in (2), the ability to invent and reason about symbols is crucial to intelligence. while we cannot understate the value of (1) , reasoning in existing symbolic system is immediately valuable (e.g. copilot).

some self-plug on my recent paper studying how people invent and communicate symbolic rules using natural language https://arxiv.org/abs/2106.07824

Linguistics is a field of study and analysis, not philosophy. And I am specifically talking about exploring the anthropological and ethnographical study of language which is where you might lose many of your assumptions. The way different languages work, how they change over time, is relevant to anyone working in NLP.

I would argue the number one fallacy of modern LLM design is people disregarding all we have come to know about language in favor of just hoping something interesting will emerge when we throw billions of parameters at it.

Also “the structure of language follows from abstracting the world into compact communicable units” is itself a “folk theory” of languages. Many supposedly neuroscientific theories of language are little more than conjecture based on assumptions.

> Intent and context are not purely symbolic.

Yup . Thats why reasoning comes in and what makes what Demis from DeepMind said make sense

Wow, thanks so much for posting this. I myself am focussed on new mathematical models for AI and this is exactly what I needed.

RemindMe! 5 days

I will be messaging you in 5 days on 2023-01-02 17:55:24 UTC to remind you of this link

5 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^(Parent commenter can ) ^(delete this message to hide from others.)

[deleted]

What are the central limitations were considering here? Let’s define them in concrete terms.

[deleted]

RemindMe! 5 days

> The reality is that AGI has become (always been?) an engineering problem.

I would not state this as fact, I'd say the majority of neuroscientists and cognitive scientists will disagree with this (or say we don't know yet), and a fair number of AI researchers would too.

I doubt any but a few researchers would be comfortable saying "Yes it's definitely an engineering problem".

I find the data efficiency argument weird. We consider a human fully formed at around twenty years of age. Is that really all that "data efficient"?

To be honest I'm not going to spend a long time thinking it through and being intellectually precise for a Reddit comment, I'd recommend you check out the AGI Debate I mentioned above for experts' opinions.

Yes, they are trained on a much larger amount of language data than a human sees in their lifetime.

However, I would argue that it's a worthwhile trade-off. Computers can more easily ingest a large amount of data. Humans get feedback from the environment (like their parents), can cross-reference different modalities, and have inductive biases.

[deleted]

Well you might wanna include the couple billion years of evolutionary selection as training time as well. Otherwise, there's a ton of stuff already "baked in" to the model.

I think the main limitations of LLMs are:

1. Hallucinations: They will make up facts.
2. Alignment/Safety: They will sometimes give undesirable outputs.
3. "Honesty": They cannot make reliable statements about their own knowledge and capabilities.
4. Reliability: They can perform a lot of tasks, but often not reliably.
5. Long-context (& lack of memory): They cannot (trivially) be used if the input size exceeds the context length.
6. Generalization: They often require task-specific finetuning or prompting.
7. Single modality: They cannot easily perform tasks on audio, image, video.
8. Input/Output paradigm: It is unclear on how to use them for tasks which don't have a specific inputs and outputs (e.g. tasks which require taking many steps).
9. Agency: LLMs don't act as agents which have their own goals.
10. Cost: Both training and inference incur significant cost.

Only #1 here really relates to their symbolic reasoning capabilities. It does imply that symbolic reasoning is a secondary objective for the models, though.

> we cannot understate the value of (1) , reasoning in existing symbolic system

ofc. and (1) may be a good way to bootstrap (2) ..

why arent we seeing more (un)supervised learning on code? perhaps with handcrafted auxiliary tasks.

when will this loop exit? how much memory will this function allocate? etc, etc. this seems to be a huge underutilized dataset.

am i missing something? (yes, its a lot of compute)

It is a conjecture that can be tested however, starting with artificial networks. I don't think it's folk theory because it s not mainstream at all

You are correct, edited to clarify that this is not consensus.