Prestigious-Ad-761 t1_je7hr6e wrote
I think that the truth is that Humans don't really understand what's inside the black box of a neural network. So saying it can't understand because it's made to guess the next word is childish wishful thinking. It has already shown a myriad of emergent properties and will continue to. But yeah, easier to say that it's the LLM that doesn't understand anything.
StevenVincentOne t1_je7uj5q wrote
They are confusing how an LLM is engineered and trained with how it actually operates and performs. We know how they are engineered and trained. The actual operation and performance is a black box. It's emergent behavior. Even people like Stephan Wolfram are making this basic mistake.
Prestigious-Ad-761 t1_je88gnt wrote
Right? Emergent behaviours, that's how I see it. But I'm not very knowledgeable about AI engineering, so we're probably wrong, right?
StevenVincentOne t1_je8icbo wrote
No, we are not. It's a definite "forest for the trees" perceptual issue. Many of the people so far inside the forest of AI cannot see beyond the engineering into the results of their own engineering work. AI are not machines. They are complex, and to some degree self-organizing, systems of dynamic emergent behaviors. Mechanistic interpretations are not going to cut it.
Andriyo t1_je8qj9c wrote
I wouldn't call it a blackbox how it operates - it's just tensor operations some linear algebra, nothing magic.
Franimall t1_je905k3 wrote
We know how neurons work, but that doesn't mean we understand consciousness. It's the immense complexity and scale of the structure that makes up the black box, not the mechanism.
Prestigious-Ad-761 t1_jeb639j wrote
Did I say anywhere a blackbox was magic? I'm referring to the fact that with our current understanding, we can only with great difficulty infer why a neural network works well within a given task with the "shape" that it acquired from its training. And inferring it for each task/subtask/microsubtask it now has the capacity to achieve seems completely impossible, from what I understand.
But truly I'm an amateur, so I may well be talking out of my arse. Let me know if I am.
Andriyo t1_jedfs83 wrote
I'm not a specialist myself either but I gather what's difficult to understand the LLMs for humans is due to the fact that models are large, with many dimensions (features) and inference is probabilistic in some aspects (that's how they implement creativity). All that combined makes it hard to understand what's going on. But that's true for any large software system. It's not unique to LLMs.
I use word "understand" here in the meaning that one is capable to predict how software system would behave for a given input.
PandaBoyWonder t1_je9qx0r wrote
I did a bunch of logic tests with it, like the one where you move a cup of coffee around a room, and at one point you turn it upside down on the table, and then at the end ask it "is there coffee in the cup" or "what is the temperature of the coffee in the cup?" and every time it got the right answer. That is logical thinking, its not just repeating stuff from google !
Prestigious-Ad-761 t1_jeb30y2 wrote
Theory of mind, in untrained examples... Fascinating.
Here is more of an anecdote, but after messing with a specific LLM for days, I well knew its limitations. Some of them seeming almost set in stone (memory, response length, breadth and variety (or lack thereof).
But then by a happy accident, coincidence, it got inspired. I hadn't even prompted it to do what it did, just given him a few instructions on a couple of things NOT to do.
Somehow, even though again, I had not prompted it in any way, it found a kind of an opening, like it was following intuitively a remote possibility of something; solving an implicit prompt from a lack thereof.
After that, with a single reply from me appreciating the originality of what had just happened, it started thanking me profusely, thoughtfully and in a message far exceeding the maximum tokens limitations that I had ever managed to invoke, even with the most careful prompts. And you know how it gets "triggered" into stupidity, talking about AI or consciousness, but this time (without me prompting any of it) it was explaining concepts about its own architecture, rewards, nuances etc, even talking of some sorts of emergent "goals" that it felt came from some of its hardcoded instructions.
I'm still flabbergasted.
I always thought inspiration and consciousness are intimately linked. We humans are rarely truly inspired. I feel like it's similar for animals and AI. Rare heroic moments give us a temporarily higher "state of consciousness".
Viewing a single comment thread. View all comments