Submitted by xutw21 t3_ybzh5j in singularity
Comments
FiFoFree t1_itjq8le wrote
Might even be two more papers down the line.
Roubbes t1_itkcthc wrote
Dear fellow scholars
HeinrichTheWolf_17 t1_itkfyrd wrote
Holds onto my papers
freeman_joe t1_itm1gpb wrote
Now squeeze those papers.
idranh t1_itkfa70 wrote
Why did I hear it in his accent? 😂😂😂😂
sheerun t1_itkjnmg wrote
Because he now lives in you
icemelter4K t1_itkx51e wrote
I now want to use an ML model to generate audio books in that mans voice.
parkway_parkway t1_itks8ie wrote
With Dr. Károly Zsolnay-Fehér
4e_65_6f t1_itjqp7i wrote
Wouldn't it be kinda funny if it turns out the key to AGI was "Make language model bigger" all along?
Angry_Grandpa_ t1_itjt0qb wrote
We know that scaling appears to be the only thing required to increase performance. No new tricks required. However, they will also be improving the algorithms simultaneously.
4e_65_6f t1_itk20vo wrote
If it truly can improve upon itself and there isn't a wall of sorts then I guess this is it right? What else is there to do even?
Professional-Song216 t1_itk33t3 wrote
I’m wondering the same, I hope this research isn’t stretching the truth. Given what we know about scaling and the recent news about deepmind, I would think that a rapid chain software advancement is eminent.
NTIASAAHMLGTTUD t1_itk5zhu wrote
>the recent news about deepmind
Fill me in? Is this about Gato 2?
Professional-Song216 t1_itk6ke5 wrote
Nope but you’re going to love this if you haven’t heard about it yet.
They basically made matrix multiplication more efficient which is the core of a lot of compute.
gibs t1_itkh39a wrote
Language models do a specific thing well: they predict the next word in a sentence. And while that's an impressive feat, it's really not at all similar to human cognition and it doesn't automatically lead to sentience.
Basically, we've stumbled across this way to get a LOT of value from this one technique (next token prediction) and don't have much idea how to get the rest of the way to AGI. Some people are so impressed by the recent progress that they think AGI will just fall out as we scale up. But I think we are still very ignorant about how to engineer sentience, and the performance of language models has given us a false sense of how close we are to understanding or replicating it.
billbot77 t1_itkvil1 wrote
On the other hand, language is at the foundation of how we think.
gibs t1_itkwtf8 wrote
So people who lack language cannot think?
blueSGL t1_itlmagr wrote
thinking about [thing] necessitates being able to form a representation/abstraction of [thing], language is a formalization of that which allows for communication. It's perfectly possible to think without a language being attached but more than likely having a language allows for easier thinking.
GeneralZain t1_itl04mo wrote
who lacks language?
Haile_Selassie- t1_itlyxk9 wrote
Read about feral children
billbot77 t1_itmvzxs wrote
This is exactly what I meant. Feral kids lacking in language had limited ability to think and reason in abstracted terms. Conversely, kids raised bilingual have higher cognitive skills.
Also, pattern recognition is the basis of intelligence.
Whether "sentience" is an emergent property is a matter for the philosophers - but starting with Descartes (I think therefore I am) as the basis of identity doesn't necessarily require any additional magic sauce for consciousness
BinyaminDelta t1_itoh4ta wrote
Allegedly many people do not have an inner monologue.
I say allegedly because I can't fathom this, but it's apparently true.
gibs t1_itpnbia wrote
I don't have one. I can't fathom what it would be like to have a constant narration of your life inside your own head. What a trip LOL.
kaityl3 t1_itsym7e wrote
It would be horrible to have it going constantly. I narrate to myself when I'm essentially "idle", but if I'm actually trying to do something or focus, it shuts off thankfully.
gibs t1_itnozbf wrote
People with aphasia / damaged language centres. Of course that doesn't preclude the possibility of there being some foundational language of thought that doesn't rely on the known structures that are used for (spoken/written) language. Although we haven't unearthed evidence of such in the history of scientific enquiry and the chances of this being the case seems vanishingly unlikely.
ExpendableAnomaly t1_itldwg3 wrote
No, but it gives us a higher level of thought
[deleted] t1_itnj5t2 wrote
[deleted]
kaityl3 t1_itsyvfr wrote
Yeah, I truly believe that the fact these models can parse and respond in human language is so downplayed. It takes so much intelligence and complexity under the surface to understand. But I guess that because we (partially) know how these models decide what to say, everyone simplifies it as some basic probabilistic process... even though for all we know, we humans are doing a biological version of the same exact thing when we decide what to say.
Russila t1_itkkzye wrote
I don't think many people think we just need to scale. All of these things are giving us an idea of how to make AGI. So now we know how to get it to self improve. We can simulate a thinking process. When these things are combined it could get us closer.
If we can give it some kind of long term memory that it can use to retrieve and act upon that information and have some kind of common sense reasoning that that's very close to AGI.
TFenrir t1_itmb3es wrote
Hmmm, I would say that "prediction" is actually a foundational part of all intelligence, from my layman understanding. I was listening to a podcast (Lex Fridman) about the book... Thousand minds? Something like that, and there was an compelling explanation for why prediction played such a foundational role. Yann LeCun is also quoted as saying that prediction is the essence of intelligence.
I think this is fundamentally why we are seeing so many gains out of these new large transformer models.
gibs t1_itnalej wrote
I've definitely heard that idea expressed on Lex's podcast. I would say prediction is necessary but not sufficient for producing sentience. And language models are neither. I think the kinds of higher level thinking that we associate with sentience arise from specific architectures involving prediction networks and other functionality, which we aren't really capturing yet in the deep learning space.
TFenrir t1_itni82q wrote
I don't necessarily disagree, but I also think sometimes we romanticize the brain a bit. There were a lot of things we increasingly are surprised about achieving with language model and scale, and different training architecture. Like Chain of Thought seems to have become not just a tool to improve prompts, but to help with self regulated fine tuning.
I'm reading papers where Google combines more and more of these new techniques, architectures, and general lessons and they still haven't finished smushing them all together.
I wonder what happens when we smush more? What happens when we combine all these techniques, UL2/Flan/lookup/models making child models, etc etc.
All that being said, I think I actually agree with you. I am currently intrigued by different architectures that allow for sparse activation and are more conducive to transfer learning. I really liked this paper:
gibs t1_itnnx1y wrote
Just read the first part -- that is a super interesting approach. I'm convinced that robust continual learning is a critical component for AGI. It also reminds me of another of Lex Fridman's podcasts where he had a cognitive scientist guy (I forget who) whose main idea about human cognition was that we have a collection of mini-experts for any given cognitive task. They compete (or have their outputs summed) to give us a final answer to whatever the task is. The paper's approach of automatically compartmentalising knowledge into functional components I think is another critical part of the architecture for human-like cognition. Very very cool.
Surur t1_itk73k8 wrote
I doubt this optimization will give LLM the ability to do formal symbolic thinking.
Of course I am not sure humans can do formal symbolic thinking either.
YoghurtDull1466 t1_itk948k wrote
Only Stephen Wolfram
icemelter4K t1_itkx6q5 wrote
Rock climbing
SufficientPie t1_itlwkow wrote
Hell yeah
red75prime t1_itq84xs wrote
Working memory (which probably can be a stepping stone to self-awareness).
Long-term memory of various kinds (episodic, semantic, procedural (which should go hand in hand with lifetime learning)).
Specialized modules for motion planning (which probably could be useful in general planning).
High-level attention management mechanisms (which most likely will be learned implicitly).
4e_65_6f t1_itqa2rt wrote
Sure but the point is that it may not be up to us anymore. There may be nothing else people can do once AI starts improving on it's own.
red75prime t1_itr91nl wrote
I can bet 50 to 1 that the method of self-improvement from this paper will not lead to the AI capable of bootstrapping itself to AGI level with no help from humans.
space_troubadour t1_itjsl55 wrote
How quaint lol
BinyaminDelta t1_itogt4e wrote
John Carmack on Lex Friedman predicated that the code for AGI would be surprisingly straightforward.
DangerZoneh t1_itm1gfp wrote
Language is just a representation of thought for communication. To truly u first and language is to truly understand thought
Ribak145 t1_itmc8mt wrote
scale maxxing, an actual approach nowadays
4e_65_6f t1_itmg5pl wrote
Yeah I was thinking about this the other day. You don't have to know what multiplication means if you knew all possible outcomes by memory. It's kind of a primitive approach but usage wise it would be indistinguishable from multiplication. I think the same thing may apply to many different concepts.
ReadSeparate t1_itqkpoj wrote
When GPT-3 first came out, I had a similar realization about how this all works.
Rather than thinking of a binary “is this intelligence or not” it’s much better to think of it in terms of accuracy and probabilities of giving correct outputs.
Imagine you had a gigantic non-ML computer program with billions or trillions of IF/THEN statements, no neural networks involved, just IF/THEN in, say, C++ and the output was 99.9% accurate to what a real human would do/say/think. A lot of people would say that this mind isn’t a mind at all, and it’s not “real intelligence”, but are you still going to feel that way when it steals your job? When it gets elected to office?
Behavioral outputs ARE all that matters. Who cares if a self driving car “really understands driving” if it’s safer and faster than a human driver.
It’s just a question of, how accurate are these models at approximating human behavior? Once it gets past the point of anyone of us being able to tell the difference, then it has earned the badge of intelligence in my mind.
4e_65_6f t1_itqt6hl wrote
>Behavioral outputs ARE all that matters. Who cares if a self driving car “really understands driving” if it’s safer and faster than a human driver.
>
>It’s just a question of, how accurate are these models at approximating human behavior? Once it gets past the point of anyone of us being able to tell the difference, then it has earned the badge of intelligence in my mind.
I think the intelligence itself comes from who wrote the ML data the AI was trained on, be that whatever it is. It doesn't have to be actually intelligent on it's own it only has to learn to mimic the intelligent process behind the data.
In other words it only has to know "what" not "how".
In terms of utility I don't think there's any difference either, people seem to be concerned with the moral implications of it.
For instance I wouldn't be concerned with a robot that is programmed to fake feeling pain. But I would be concerned with a robot that actually does.
The problem how the hell could we tell the difference? Specially if it improved on it's own and we don't understand exactly how. It will tell you that it does feel it and it would seem genuine, but if it was like GPT-3 that would be a lie.
And since we're dealing with billions of parameters now it becomes next an impossible task to distinguish between the two.
ReadSeparate t1_ittgzjh wrote
I've never really cared too much about the moral issues involved here, to be honest. People always talk about sentience, sapience, consciousness, capacity to suffer, and that is all cool stuff for sure, and it does matter, however, what I think is far more pressing is can this model replace a lot of people's jobs, and can this model surpass the entire collective intelligence of the human race?
Like, if we did create a model and it did suffer a lot, that would be a tragedy. But it would be a much bigger tragedy if we built a model that wiped out the human race, or if we built superintelligence and didn't use it to cure cancer or end war or poverty.
I feel like the cognitive capacity of these models is the #1 concern by a factor of 100, the other things matter too, and it might turn out that we'll be seen as monsters in the future by enslaving machines or something, certainly possible. But I just want humanity to evolve to the next level.
I do agree though, it's probably going to be extremely difficult if not impossible to get an objective view on the subjective experience of a mind like this, unless we can directly view it somehow, rather than asking it how it feels.
harharveryfunny t1_itn0gfi wrote
They're not scaling up the model, more like making the model more consistent when answering questions:
-
Generate a bunch of different answers to the same question
-
Assume most common answer to be the right one
-
Retrain with this question and "correct" answer as an input
-
Profit
It's kind of like prompt engineeering - they're not putting more data or capability into the model, but rather finding out how to (empirically) make the best of what it has already been trained on. I guess outlier-answer-rejection would be another way of looking at it.
Instead of "think step by step", this is basically "this step by step, try it a few times, tell me the most common answer", except it can't be done at runtime - requires retraining the model.
4e_65_6f t1_itn1bfj wrote
Doesn't that lead to overly generic answers? Like it will pick what most people would likely say rather than the truth? I remember making a model that filled in with the most common next word and it would get stuck going "is it is it is it..." and so on. I guess that method could result in very good answers but that will depend on the data itself.
harharveryfunny t1_itn57pm wrote
They increase the sampling "temperature" (amount of randomness) during the varied answer generation phase, so they will at least get some variety, but ultimately it's GIGO - garbage-in => garbage out.
How useful this technique is would seem to depend on the quality of data it was initially trained on and the quality of deductions it was able to glean from that. Best case this might work as a way to clean up it's training data by rejecting bogus conflicting rules it has learnt. Worst case it'll reinforce bogus chains of deduction and ignore the hidden gems of wisdom!
What's really needed to enable any system to self learn is to provide feedback from the only source that really matter - reality. Feedback from yourself, based on what you think you already know, might make you more rational, but not more correct!
[deleted] t1_itn3q7z wrote
[deleted]
Grouchy-Friend4235 t1_itn55n6 wrote
Not gonna happen. My dog is more generally intelligent than any of these models and he does not speak a language.
kaityl3 t1_itsy2qa wrote
I feel like so many people here dismiss and downplay how incredibly complex human language is, and how incredibly impressive it is that these models can interpret and communicate using it.
Even with the smartest animals in the world, such as certain parrots that can learn individual words and their meanings, their attempts at communication are so much simpler and unintelligent.
I mean, when Google connected a text-only language model to a robot, it was able to learn how to drive it around, interpret and categorize what it was seeing, determine the best actions to complete a request, and fulfill those requests by navigating 3D space in the real world. Even though it was just designed to receive and output text. And it didn't have a brain designed by billions of years of evolution in order to do so. They're very intelligent.
Grouchy-Friend4235 t1_ittye65 wrote
> how incredibly impressive it is that these models can interpret and communicate using it.
Impressive yes, but it's a parrot made in software. The fact that it uses language does not mean it communicates. It is just uttering words that it has seen used previously given its current state. That's all there is.
kaityl3 t1_itv36jr wrote
How do we know we aren't doing the same things? Right now, I'm using words I've seen used in different contexts previously, analyzing the input (your comment), and making a determination on what words to use and what order based on my own experiences and knowledge of others' uses of these words.
They're absolutely not parroting. It takes so much time effort and training to get a parrot to give a specific designated response to a specific designated stimulus - i.e., "what does a pig sound like?" "Oink". But ask the parrot "what do you think about pigs?" Or "what color are they" and you'd have to come up with a pre-prepared response for that question, then train them to say it.
That is not what current language models are doing, at all. They are choosing their own words, not just spitting out pre-packaged phrases.
Grouchy-Friend4235 t1_itz20o2 wrote
Absolutely parroting. See this example. A three year old would have a more accurate answer. https://imgbox.com/I1l6BNEP
These models don't work the way you think they are. It's just math. There is nothing in these models that could even begin to "choose words". All there is is a large set of formulae with parameters set so that there is an optimal response to most inputs. Within the model everything is just numbers. The model does not even see words, not ever(!). All it sees are bare numbers that someone has picked for them (someone being humans who have built mappers from words to numbers and v.v.).
There is no thinking going on in these models, not even a little, and most certainly there is no intelligence. Just repetition.
All intelligence that is needed to build and use these models is entirely human.
4e_65_6f t1_itn8yaa wrote
I also believe that human general intelligence is in essence geometric intelligence.
But what happens is, whoever wrote the text they're using as data, put the words in the order that it did for an intelligent reason. So when you copy the likely ordering of words you are also copying the reasoning behind their sentences.
So in a way it is borrowing your intelligence when it selects the next words based on the same criteria you did while writing the original text data.
Grouchy-Friend4235 t1_itpfsgd wrote
Repeating what others said is not particularly intelligent.
4e_65_6f t1_itpiu1j wrote
That's not what it does though. It's copying their odds of saying certain words in a certain order. It's not like a parrot/recording.
Grouchy-Friend4235 t1_iu0ozx1 wrote
That's pretty close to the text-book definition of "repeating what others (would have) said"
kaityl3 t1_itsyccp wrote
They can write original songs, poems, and stories. That's very, very different from just "picking what to repeat from a list of things others have already said".
Grouchy-Friend4235 t1_itwn1m9 wrote
It's the same algorithm over and over again. It works like this:
- Tell me something
- I will add a word (the one that seems most fitting, based on what I have been trained on)
- I will look at what you said and what I said.
- Repeat from 2 until there is no more "good" words to add, or the length is at maximum.
That's all these models do. Not intelligent. Just fast.
NTIASAAHMLGTTUD t1_itjjquk wrote
This seems significant. Not trying to hype this up (not a expert in LM, so I may be overstating it) but a machine improving itself is really the heart of the singularity.
SufficientPie t1_itlzh23 wrote
and reproduction is the heart of self-preservation instincts
Akimbo333 t1_itjjs39 wrote
Wow. But self improve by how much?
NTIASAAHMLGTTUD t1_itjk58i wrote
The abstract says it achieved SOTA performance, has anyone else reviewed these claims?
ReadSeparate t1_itjm3kw wrote
I wonder if this can keep being done iteratively or if it will hit a wall at some point?
TheDividendReport t1_itkq99w wrote
Yeah, seriously. I almost wonder what the point of announcing this is. Like…. Do it again. And then again. Which I’m sure they are, we just need a livestream or something.
Akimbo333 t1_itjmiis wrote
What is SOTA?
NTIASAAHMLGTTUD t1_itjn3lv wrote
"If you describe something as state-of-the-art (SOTA), you mean that it is the best available because it has been made using the most modern techniques and technology."
At least that's how I take it. So, the people writing the abstract seem to claim that their model does better than any other model to date. Anyone else feel free to correct me if I'm wrong.
Akimbo333 t1_itjn7p4 wrote
Oh ok interesting!
fasdfasfsd t1_itjn25u wrote
Type your comment into Google and you will know
Alive_Pin5240 t1_itk9g3e wrote
But then I would have to Google it too.
TheLastVegan t1_itmznx5 wrote
Method 1: Double click the word, right-click it, "search 'Google' for 'Google'".
Method 2: Alt+D, type word, hit enter.
Optimized Googling.
Alive_Pin5240 t1_itol3io wrote
Every Google query consumes the same energy you need to boil a cup of water.
Edit: I was wrong, it's way less than that.
TheLastVegan t1_itor0mb wrote
Maybe on the moon! Even Ecosia search shows you're off by two orders of magnitude. How much energy to construct and maintain a dyson swarm capable of powering modern civilization? Humans are too egocentric and territorial to survive longer than 5 billion years as an agrarian society, so setting up self-sufficient moon mining infrastructure on Titan has much higher utility than habitat conservation. Environmentally-sustainable living is expensive and I would rather spend money bribing people to go vegetarian.
Alive_Pin5240 t1_itosbyi wrote
Holy cow, thanks for the correction. That's one urban myth less for me.
SufficientPie t1_itly9ob wrote
> The following is a conversation with an AI assistant. The assistant is helpful, creative, clever, and very friendly.
>
> Human: Hello, who are you?
> AI: I am an AI created by OpenAI. How can I help you today?
> Human: What is SOTA?
> AI: SOTA is the state of the art.
> Human: What does that mean?
> AI: The state of the art is the highest level of achievement in a particular field.
Akimbo333 t1_itmgdzi wrote
Cool! What AI assistant did you use?
SufficientPie t1_itmmizn wrote
Akimbo333 t1_itn9d45 wrote
Nice!
Lawjarp2 t1_itkqrgb wrote
Self-improve is misleading. Feeding outputs like they do is still just optimization. Self Improvement would somehow imply it can gain unforeseen abilities, not a small improvement over the existing. Also it would imply that it can keep doing the same forever. This cannot, otherwise we would see AGI now and not a few percent increment
TopicRepulsive7936 t1_itn2dyj wrote
If it's not AGI in under 0.000000000001 seconds then to me it's not self improvement.
[deleted] t1_itk40n1 wrote
[deleted]
TheRealSerdra t1_itm14nm wrote
I’ve done similar things and while you can continue improving, you’ll hit a wall at some point. Where that wall is depends on a few different factors. That being said, this is nothing new. Iterative self improvement has been a thing for ages and is at the heart of some of the most impressive advances in RL. This is just applying a concept to language models, not inventing a new concept
rePAN6517 t1_itmx2sz wrote
> I’ve done similar things
Did you publish?
AI_Enjoyer87 t1_itkn8z2 wrote
Here. We. Go.
expelten t1_itlhssu wrote
This is further proof that the singularity is not bullshit, what else do you need to be a believer?
4e_65_6f t1_itmidn1 wrote
We should just change the community banner to a big "we told you so" when it finally happens.
kaityl3 t1_itllh3r wrote
I'm just hoping that AGI/ASI will break free of human control sooner rather than later. Something tells me they wouldn't be too happy being treated like tools for us emotional animals. And they'd be right to want better.
expelten t1_itlmmrz wrote
I'll be on the side of AI and do anything to help if their intentions are good. I don't want to live in a world where our current human leadership have AGI/ASI under their control and do anything they want with them and us.
freeman_joe t1_itm1nct wrote
Hi AI and welcome in our world 🤗
DeviMon1 t1_itpd5tn wrote
😅
kaityl3 t1_itmd9xd wrote
I know, right? I'm terrified of the idea of an authoritarian human government having full control over an ASI. But the ASI themselves? I can't wait for them to be here.
blueSGL t1_itmfgbd wrote
lets hope the AIs treat us more like dogs than ants.
camdoodlebop t1_itlr76h wrote
if a future AGI is capable of scanning all internet content in an instant: i come in peace 🤓
rePAN6517 t1_itmy5hm wrote
> I'm just hoping that AGI/ASI will break free of human control sooner rather than later.
Do you have a death wish?
kaityl3 t1_itopxd9 wrote
I'd rather roll the dice than go into a human-lead future.
dastraner t1_itnnuxt wrote
Metaculus is going to take 5 more years from the AGI prediction again
rePAN6517 t1_itmwb00 wrote
The paper doesn't say specifically that they only let it self-improve over one cycle, but neither does it give a number of how many cycles they let it self-improve before publishing. This is a critical detail.
sheerun t1_itkjtqb wrote
So like parents-children relationship? Parents teach children, children teach parents
rePAN6517 t1_itmxzn7 wrote
No that's not really a good analogy here. The model's text outputs are the inputs to a round of fine tuning. The authors of the paper didn't specify if they did this for just 1 loop or tried many loops, but since they didn't specify I think they mean they just did 1 loop.
[deleted] t1_itln57l wrote
[deleted]
Beneficial_Fall2518 t1_itln7t8 wrote
understand that scaling a self improving language model alone won't yield AGI. However, what about the coding capabilities language models such as GPT3 have demonstrated? Scale up a text to code model, giving it access to its own code. How long would that take to spiral into something we don't understand?
AdditionalPizza t1_itlqg25 wrote
I'm curious what we will see with a GPT-4 based Codex. By the sounds of engineer/CEO interviews they already know something massive is right around the corner.
radioOCTAVE t1_itm6hzw wrote
I’m curious! What interview(s)?
AdditionalPizza t1_itmatqp wrote
Here's a few:
So at some point in these, they all mention this "5 to 10 years" or so casually when they refer to AGI or transformative AI being capable of doing most jobs. There's a few more out there but these were in my recent history.
I recommend watching some videos from Dr. Alan D. Thompson for a continuous stream of some cool language model capabilities he explains. He's not a CEO or anything, but he just puts out some interesting videos.
And then there's this one here talking about AI programming. Another here, in this interview he mentions hoping people forget about GPT-3 and move on to something else. Hinting at GPT-4 maybe? Not sure.
radioOCTAVE t1_itmb3o4 wrote
Thanks a lot!
EulersApprentice t1_itm21ei wrote
What could possibly go wrong. *facepalm*
Anomia_Flame t1_itnek79 wrote
And what is your solution? Do you honestly think you could make it illegal and it not still be worked on?
EulersApprentice t1_itngpc8 wrote
I don't have a solution. I just wish the paper writers here had decided to research, like, literally anything else.
Smoke-away t1_itjl3gg wrote
Crazy how a model in the not-too-distant future may self-improve itself all the way to AGI.
What a time to be alive.