Some people mistakenly believe that the technological singularity is synonymous with AGI and AGI by 2025-2030 is unlikely but not impossible.

The technology to do all the individual steps is already there. It's just a matter of doing it yourself or waiting for some business to do the work for you. I expect such a service will be a available later this year either as a standalone or built in to YouTube Premium.

Going from O(n^2) to O(nlog(n)) for context window size let's you have a context window of 1.3 million tokens using the same space needed for GPT-3's 8000 tokens.

Yes but how many parameters must you actually have to store all the knowledge you realistically need. Maybe a few billion parameters is enough to store the basics of every concept known to man and more specific details can be stored in an external file that the neural net can access with API calls.

Humans don't suffer from overfitting if they train on the same data too much.

It was shown recently that for LLMs ~0.01% of parameters explain >95% of performance.

https://www.reddit.com/r/artificial/comments/115jht7/prompt_engineer_jobs_paying_300k_w_no_degree/

>Singularity defines the point in a function where it takes an infinite value,

It doesn't need to be infinite it can also be undefined or otherwise not well-behaved. For instance the function 1/x is never infinite for any finite value of x but it has a singularity because it is undefined at x=0. Another example is the piecewise function F(x)={x^2 if x>=0 {-x^2 if x<=0 for which x=0 has a definite value of y=0 but x=0 is considered a singularity for the purposes of calculus because it is not differentiable at that point.

The word singularity comes from math where it means a part of a graph where a function becomes discontinuous (i.e. the value of the line at that point is either undefined or infinite). Things like evolution, immortality, and fusing together into a hive mind have nothing to do with it, at least not inherently.

The idea of the technological singularity comes from the observation that the rate of new technological advances is becoming faster over time such that our total level of technology is growing hyperbolically. If human technology continues growing at the current hyperbolic rate or something close to it then relatively soon we will reach the singularity on the graph of new technological innovations which is the point in time where the rate of new technological innovation will become infinite.

The general assumption of this subreddit is that the technological singularity occurs as the result of the creation of a self improving AI which will then proceed to rapidly create better and better versions of itself. The hope is that if AI is controllable or at least benevolent it could bring about a golden age where all science is known and essentially all economic activity is automated.