Is than an euthemism for unemployment?

I think one of the thigns that will indicate that singularity is going well will be the collapse of the USA (and most other countries) political systems. Because the hinge on the people being ignorant, and AI will change that.

There was nothing about AGI in the original post.

Actually, Matrix could be the case of the relatively GOOD AI. Yes, it disempowers the humanity, but keeps us safe, gives us entertainment and an illusion of purpose.

Also, looks like in none of the good/optimistic scenarios singularity has happened.

I think it's not that AI is bad at math specifically. It's just that math is the easiest way to formulate a compact question that requires a non-trivial precise solution.

Unironically, The Matrix. As an example of how BENIGN superintelligence will treat humanity.
As humanity will be completely useless after the singularity, lying in pods and being entertained in an illusion is probably the best what humans will be able to do.

I'm for democratization of control, but not for anarchy of control. I.e. don't hand out models to everybody, but regulate the process of AI deployment and development democratically.

I haven't noticed any improvement in memory requirements for 5 months on Stable Diffusion... My RTX2060 still is enough for 1024x640, but not more.

LLaMA does good on tests on small models, but small size could make it not as fit for RLHF.

There is also miniaturisation for inference by reducing precision to int8 or even4. But that does not fit for training, and I believe AGI requires real-time training.

So, in theory, AGI could be achieved even without big "a-ha"-s. Take existing training methods, train on many different domains and data architectures, add tree earch from AlphaGo and real time training - and we probably will be close. But it would require pretty big hardware. And would be "only" superhuman in some specific domains.

People starting to use it for evil. Frauds, terrorism, etc.

Not exactly. There are methods to analyse the LLM to figure, say, which "neurons" do what. But they are quite undeveloped still

https://alignmentjam.com/post/quickstart-guide-for-mechanistic-interpretability

You mean, OpenAI was taught on the texts that had way more anti-disabled hate than anti-republican hate? Where have they found them?

How do you know it's meaner? Is it avaiable through API already?

That depends on if there is some new revolutionary breakthrough. They are hard to predict. But considering how many people will research the field, they are quite likely.

If time travel or FTL travel is not possible by the laws of physics, it's not possible. No amount of intelligence can change it.

Problem with that approach is that 1. we don't know how to do that reliably and 2. by the time AGI will be invented, it will likely to be able run on home computer or network of those, and there will be someone evil or reckless enough to run it without the handlebars.

It seems that it is close to SOTA on 60-70B models. "Only" big deal is that the smaller LLAMA models show results comparable to much bigger SOTAs.

Speaking of that, I haven't noticed much algorithmic improvement in past couple of decades, except maybe some niche cases. If anything, less optimised algorithm are used now, because hardware can handle that.

Then famirializing oneself with Alignment issue could be a good early step on the way to ASI research.
This https://www.lesswrong.com/posts/Aq82XqYhgqdPdPrBA/full-transcript-eliezer-yudkowsky-on-the-bankless-podcast could be a good introduction, imho.

We only know of the capabilities of AIs that are published. There is the not-zero probability that someone has already figured out and implemented AGI on the farm of GPUs bought from miners, for example.

Have you solved the Alignment problem?

It's possible that it has already happened.

Yes. Made by people not happy with Character.ai nerfs.

https://rentry.org/pygmalion-ai

Then look at the direction of the https://rentry.org/pygmalion-ai

Question is, was "entitled 14 year old on tumblr" behaviour invented by AI from scratch, or it's just mimicking the behaviour of the actual "entitled 14 year old on tumblr" from the training set?