How do you go from the LLMs of today to full blown AGI in 6 years?

Transitioning from an AI that only responds to prompt stimulus, to an AI that can take initiative of its own accord. That might still turn out to be surprisingly hard

You find a way to make it recurrent (keep state alongside input buffer), add memory (working, as a part of the said state, and long-term), overcome catastrophic forgetting in online learning, find efficient intrinsic motivations. Maybe it's enough.

2035+

The one AGI project I'm close to has a design, potential solutions for all the big problems, and a loose plan for implementation. So I'm going largely off of that, but funding, building, training, and testing takes time. Rushing it wouldn't help anything anyway.

The few others that I've seen that have potential (in my opinion of course) will probably get it eventually, but are missing some things. Whether those things become showstoppers or not has yet to be seen. And no, they have nothing to do with LLMs.

I also think that society needs to prepare. I'm actually becoming more comfortable with people calling non-AGI AGI because it will help people get used to it, and encourage discussion, get new laws on the books, etc. I don't think there's much use trying to pin an exact date on it, because even after the first real AGI is available, it will just be the first of many.

I expect AGI around 2030.

I think that the most likely reason (but still not sufficiently probable to affect my estimate) for extending AGI timeline is that the brain does use quantum acceleration for some parts of its functionality.

Do you consider my predictions made at the start of 2023 to be pessimistic or conservative?

2030 - AGI

2040 - ASI

2050 - Singularity

Of course I hope I’m wrong and that these things happen a lot sooner. The sooner the better.

It’s just I don’t like to be too optimistic to the point to where it’s wishful thinking.

The focus is currently on Deep Learning. So why would DL not bring AGI in its current form? First in simple terms, how does it work? The most common setup uses inputs and weights. The sum of the products is propagated. There are ReLus, batch normalisation, residual connections and all kinds of tricks in between. The outputs are checked against expected values. Weights are then updated to fit the expected outputs against given inputs.

There are multiple neural layers. That is why we speak of Deep Learning. So to use a crude analogy, imagine that you are the leader of a squad. Imagine that your soldiers understand 80% of your orders. Now imagine being the platoon leader. Imagine again that your squad leaders again understand 80% of your orders. How many of your orders reach your soldiers? Imagine having a hundred or more layers. Adding layers isn't free. And with almost all AI companies doing the same, we will run out of GPUs soon.

Also real neurons are more complicated than in the DL models. There are things like spiking, brain plasticity, neurotransmitters, and synapse plasticity that DL doesn't take into account. So the obvious solution is neuromorphic hardware and appropriate algorithms. It's anyone's guess when they will be ready.

How do you plan to make it not kill everyone, either by mistake in alignment, or someone intentionally running a misaligned AGI? I don't see how it can be done without extreme safety measures, such as many AIs and people keeping an eye on every AI and evey human at all time.

Pretty easily...?

That is clearly the obvious next step.

I'm not sure how easy that will be 'tho.

It could be that a large 'frozen' model in combination with some clever run-time code and a modicum of short/medium term memory would suffice.

After all, the human brain seems (to me) to be a huge static memory plus relatively little run-time stuff.

I've no idea about the AGI timeline.

However, we are very close to quasi AGI.

We could have 99% human-like help desk, human like doctors etc within a year or so.

Backpropagation is a tool that takes care of servicemen not getting the orders. There's the vanishing gradient problem affecting deep networks, but RELUs and residual connections seem to take care of it just fine. Mitigation of the problem in recurrent networks is harder though.

As for the brain... The brain architecture most likely is not the one and only architecture suitable for general intelligence. And taking into account that researchers get similar results when scaling up different architectures, there are quite a few of them.

Kurzweil's prediction is based on two parameters:

1. The availability of computing power sufficient to simulate a human brain.
2. Neuroscience being advanced enough to tell us how to simulate a human brain at a scale sufficient to produce intelligence.

I don't think that Kurzweil does a bad job at ballparking the calculations per second of the brain. His estimate is under today's top supercomputers, but still far greater than a typical desktop workstation. (If I'm doing my math right, it would take something like 2,000 Nvidia GeForce 4090s to reach Kurzweil's estimate at double precision, which is the precision supercomputers are measured at, or ~28 at half or full precision.)

That leaves us with the neuroscience. I'm not a neuroscientist, but I am another kind of life scientist. Computing power has followed this accelerating trend, but basic science is a lot slower. It is more of a punctuated equilibrium model than an exponential. Things move really fast when you know what to do next, and then it hits a roadblock while you make sense of all this new information you gather. It also relies on funding, and people. Scientists at the Human Brain Project consider real-time models a long term goal. Static, high resolution models that incorporate structure and other data (genomics, proteomics, etc.) are listed as a medium term goal. I don't know what "long term" is to this group, but I'm assuming it's more than 6 years. And if all that complexity is required, then Kurzweil is likely off by several orders of magnitude, which could put us decades out from his prediction. Then again, maybe you don't need to model everything in that great of detail to get to intelligence, but that goes against Kurzweil's prediction.

Of course, this all presupposes that you need a human brain for human-level intelligence. It's not a bad guess, as all things that we know to be intelligent have nervous systems evolved on Earth and share some last common ancestor. If we go another route to intelligence, that puts us back at factoring people into the process. We either need people to design this alternate intelligence architecture, or create weak AI that's capable of designing this other architecture.

While I could be wrong, and maybe you can slap some additional capability modules onto an LLM, let it run and retrain itself constantly on a circa 2029 supercomputer, and that will be sufficient. But I A) don't know for sure that will be the case, and B) think that if it does happen, it's kind of just a coincidence and not to the letter of Kurzweil's prediction.

Their approach to safety, to put it simply, would be to keep it in an invisible box, watched by an invisible guard that intervenes covertly when needed to keep it within that box should it stray towards the outside.

You are right in that AIs and people are going to have to watch out for other people and their AIs. But even if you remove the AI component, you can still say the same. Some people will try to scam you, take advantage of you, use you, or worse. AI makes that quicker and easier, so we'll have to be on the lookout, we'll have to discuss these things, and we'll have to prepare and create laws anticipating these things. But if everyone can gain access to it equally, either as SAAS or open source and locally run, then there will be tools to protect against malicious uses. That's all that can be done really, and no one company will be able to solve that.

An AI that only responds to prompts and an AI that has " will of its own" are the same thing. If you train a model to mimic the behaivor of a dog, for a outside observer it will look like it has developed some kind of initiative. The human behaivor can be expressed as a sequence of tokens, thus, you can train a model to predict the next action given context.

Yeah I totally agree, I don't really believe we won't have AGI until after 2029 but it was the only counter example I could think of

What we might need is neural data, just wait until Neuralink release an dataset of 2 years of neural activity gathered from 2000 patients. With that you can train a diffusion model to generate brains.

> Their approach to safety, to put it simply, would be to keep it in an invisible box, watched by an invisible guard that intervenes covertly when needed to keep it within that box should it stray towards the outside.

Can't ideas still leak out and get into human minds?

If we don't have a robust safery system that works acroos the companies and across the states by that time, I don't see how we will survive that.

Can you give an example?

We don't have a lot of time, but we do have time. I don't think there will be any immediate critical risks, especially with safety in mind, but what risk there is might even be mitigated by near-future AI. chatGPT for example may soon enough be adequate in fact-checking misinformation. Other AIs might be able to spot deepfakes. It would help if more people started discussing the ways AGI can potentially be misused, so everybody can begin preparing and building up protections.

Most of our rules and conventions are extremely arbitrary, highly suboptimal, and maintained via cultural conditioning.

Do you really expect for ChatGPT to go against the USA machine of disinformation? Do you think it will be able to give a balanced report on controversial issues, taking in account the credibility and affiliation of sources, and quality of reasoning (such as NOT taking into account the "proofs" based on "alleged" and "highly likely"). Do you think it will honestly present the point of views from countries and sources not affiliated/bought by USA and/or Dem or Rep party? Do you think it will let the user define the criteria for credibility by him/herself and give info based on that criteria, not push the "only truth"?

Because if it won't, and AI will be used as a way of powers to braiwash the masses, instead as a power for masses to resist brainwahsing, then we'll have very gullible population and very dishonest AI by the time it will matter the most.

P.S. And yes, if/when China or Russia will make something like ChatGPT, it will probably be pushing their government agendas just like ChatGPT pushes US agenda. But is there a hope for impartial AI?

I mean, that's out of their hands and mine. I probably shouldn't have used chatGPT as an example, I just mean near-future narrow AI. It's possible we'll have non-biased AI over the next few years (or minimally biased at least), but nobody can tell how many and how effective they'll be.

The guard would be a compartmentalized hybridization of the overall AGI system, so it too would have a generalized understanding of what bad undesirable things are, even according to our arbitrary framework of cultural conditioning. So could undesirable ideas leak out? Well, no not really. Not if the guard and other safety components are working as intended, AND if the guard is programmed with enough explicit rules and conditions and enough examples to effectively extrapolate from (meaning not every case needs to be accounted for if patterns can be derived).

How do you handle risk that emerges years after something becomes well known and popular? Let's say it produces an idea that starts out safe but then mutates? Or, a person merges two objectively safe (on their own) AGI-produced ideas, producing a dangerous one (that could not have been achieved without AI/AGI)?

I dunno, I have the feeling there's a lot of unknown unknowns and likely some (yet to be discovered) incorrect "knowns" floating out there.

If it does it only means that quantum computation is really easy.

>a person merges two objectively safe (on their own) AGI-produced ideas

Well that's kind of the real problem isn't it? A person, or people, and their misuse or misinterpretation or whatever mistake they're making. You're talking societal problems that no one company is going to be able to solve. They can only anticipate what they can, hope the AGI anticipates the rest, and future problems can be tackled as they come.

It could still require considerable time to get enough hints on how the brain does that to implement it technologically. Evolutionary solutions can be messy.

[deleted]

The argument I find most convincing is that whilst technology is moving fast, politicts is always slow. Stuff like regulations and a general political misunderstanding of AI might lead to pushback against AI or they might purposely slow down development. Even if AI was made that could replace something such a mcdonalds workers the government may place a ban or tax on companies using AI in order to keep employment high. It sounds like a dumb solution (and I agree) but to politicians this might sound like a good solution.

I believe a capability like that could be a key for our survival. It is required for our Alignment as the humanity. I.e. us being able to act together for the interest of Humanity as a whole. As the direst political lies are usually aimed at the splitting people apart and fear each other, as they are easier to control and manipulate in that state.
Also, this ability could be necessary for strong AI even being possible, as strong AI should be able to reason successfully on partially unreliable information.
And lastly, this ability will be necessary for AIs to check each other AIs reasoning.

This is true even without AI, and it seems we weren't ready (climate change) even for the technology we developed so far.

Is the invisible guard another AGI? Does it has its own guard?

Kurzweil has put AGI as surpassing human intelligence as 2045.

2029 is the date he put as Ai passing the turning test. I wouldn't be surprised if Chat-GPT can already pass the turning test.

Personally I agree with his dates and I also agree with him that it may happen sooner.

In 1990 Kurtzweil predicted an Ai would beat the worlds best chess player by 2000. It occurred in 1997.

I see no reason why understanding the human brain would be needed.

We have more than enough concepts and AGI models, we just need more compute imho. Compute (for the same cost) increases by a thousand times every ten years. So by Kurzweils 2045 date, compute for the same cost can be estimated to be 4.2 million times more than today.

Even if moors law ended the trend would continue because of the fact that server farms are growing at an exponential pace, and solar energy is dropping towards zero. If we have a breakthrough in fusion power it will accelerate beyond our models.

Today we can simulate vision (roughly 20% of the human brain) but we're simulating it in a way that's far more computationally efficient than the human brain, because we're making the absolute most out of our hardware.

It's pretty likely we'll reach super human level AGI well before 2045.

I don't think that you have to simulate a human brain to get intelligence, either. I discuss that toward the end of my comment. But the OP asked about counterarguments to the Kurzweil timeline for AGI. Kurzweil explicitly bases his timeline on the those two factors: computing power and a good enough brain model to simulate in real time. I don't think that the neuroscience will be there in 6 years to meet Kurzweil's timeline.

If we get AGI in 2029, it will likely be specifically because some other architecture does work. It won't be because Kurzweil was correct. In some writings, Kurzweil goes further and says that we'll have this model of the brain, because we'll have this really amazing nanotech in the late 2020s that will be able to non-invasively map all the synapses, activation states of neurons, etc. I'm not particularly up on that literature, but I don't think we're anywhere close to having that tech. I expect that we'll need AGI/ASI, first, to get there before 2100.

With regards to your own thinking, you only mention computing power. Do you think that intelligence is emergent given a system that produces enough FLOPS? Or do you think that we'll just have enough spare computing power to analyze data, run weak AI, etc., that will help us discover how to make an AGI? I don't believe that intelligence is emergent based on processing power, or else today's top supercomputers would be AGIs already, as they surpass most estimates of the human brain's computational capabilities. That implies that architecture is important. Today, we don't really have ideas that will confidently produce an AGI other than a simulated brain. But maybe we'll come up with a plan in the next couple of decades. (I am really interested to see what a LLM with a memory, some fact-checking heuristics, ability to constantly retrain, and some additional modalities would be like.)

A hybridized partition of the overall system. It uses the same cognitive functions, but has separate memory, objectives, recognition, etc. They hope for the whole thing to be as modular and intercompatible as possible, largely through their generalization schema. So one segment of it will have personality parameters, goals, memory, and whatever else, and the rest will be roughly equivalent to subconscious processes in the human brain, which will be shared with the partition. As I understand it, the guard would be strict and static, unless it's objectives or parameters are updated by the user via natural language programming. So it's actions should be predictable, but if it somehow deviates then the rest of the system should be able to recognize it as an unexpected thought (or action or whatever), either consciously or subconsciously, which would feedback to the guard and reinitialize it, like a self-correcting measure. And once it has been corrected, it can edit the memory of the main partition so that it's unaware of the fault. None of this has been tested yet, and they're still revising some things, so this may change in the future.

None of those people know, the people you think are genius are only one person at the head of a big team. Progress isn't made by one person, its always always always a group thing. So while they may know a lot in their field, they also for sure don't know a lot. Other people do, and with something as complex as this there's just no way to know since there's no collective tracking of who is doing what where