Don't forget qualia. That's another one of the possibilities that they could mean but aren't using the right word for. Though usually it's just a nebulous thing, no specific definition for sentience. Sentience is usually used to mean "is similar to me".

> For the chatbot thing, why can't the LLM generate a non-displayed output, "test it", and try again

You can! There are systems designed around that. OpenAI even internally had GPT-4 using a multi-stage response system (a read-execute-print loop, they called it) while testing, to give it more power. There is also the "Reflexion" posts on this sub lately, where they have GPT-4 improve on its own writing. But, A, it's too expensive. Using a reflective system means lots of extra words, and each word costs more electricity.

And B, LLMs currently love to get sidetracked. They use the word "hallucinations" to say that the LLM just starts making things up, or acting like you asked a different question, or many other things. Adding an internal thought process dramatically increases the chances of LLMs going off the rails. There are solutions to this (usually, papers on it will describe their solutions as "grounding" the AI), but once again, they cost more money to do.

So that's why all these chatbots aren't as good as they could be. it's just not worth the electricity to them.

Remember, the text of an LLM is literally the thought process of the LLM. Trying to have it instantly write an answer to what you ask makes it nigh impossible to accomplish the task. Microsoft and OpenAI have said that the chatbot format degrades the AI's intelligence, but it's the format that is the most useful/profitable currently. If a human were to try to write a sentence with 8 words, they'd mentally retry multiple times, counting over and over, before finally saying an 8 word sentence. By using a chat format, the AI can't do this.

ALSO, the AI does not speak English. It gets handing a bunch of vectors, which do not directly correspond to word count, and it thinks about those vectors, before handing back a number. The fact these vectors + a number directly translate into human language doesn't mean it's going to have an easy time figuring out how many vectors add up to 8 words. That's just a really hard task for LLMs to learn.

Unfortunately, they didn't tell us anything about how they did the codeforces test. It sounds like they just tried zero-shot, had GPT-4 see the problem and immediately write code to solve it. But that's not humans solve codeforces problems, we sit down and think through the problem. In a more real world scenario, I think GPT-4 would do way better at codeforces. Still not as good as a human, but definitely way better than their test.

(I believe in a hard take off, so this is just a thought experiment)

If you care about the difference between human vs AI, then it's not the product you are after, but the production, the creation of the art, right? So paying for a product doesn't matter. Instead, there could be a niche where you pay to watch an artist work and apply themselves creatively.

As the article says, Microsoft just gave them $10 billion.

/u/rationalkat made a cherry-picked list in the big predictions thread:

• Rob Bensinger (MIRI Berkeley)
  ----> AGI: ~2023-42
• Ben Goertzel (SingularityNET, OpenCog)
  ----> AGI: ~2026-27
• Jacob Cannell (Vast.ai, lesswrong-author)
  ----> AGI: ~2026-32
• Richard Sutton (Deepmind Alberta)
  ----> AGI: ~2027-32?
• Jim Keller (Tenstorrent)
  ----> AGI: ~2027-32?
• Nathan Helm-Burger (AI alignment researcher; lesswrong-author)
  ----> AGI: ~2027-37
• Geordie Rose (D-Wave, Sanctuary AI)
  ----> AGI: ~2028
• Cathie Wood (ARKInvest)
  ----> AGI: ~2028-34
• Aran Komatsuzaki (EleutherAI; was research intern at Google)
  ----> AGI: ~2028-38?
• Shane Legg (DeepMind co-founder and chief scientist)
  ----> AGI: ~2028-40
• Ray Kurzweil (Google)
  ----> AGI: <2029
• Elon Musk (Tesla, SpaceX)
  ----> AGI: <2029
• Brent Oster (Orbai)
  ----> AGI: ~2029
• Vernor Vinge (Mathematician, computer scientist, sci-fi-author)
  ----> AGI: <2030
• John Carmack (Keen Technologies)
  ----> AGI: ~2030
• Connor Leahy (EleutherAI, Conjecture)
  ----> AGI: ~2030
• Matthew Griffin (Futurist, 311 Institute)
  ----> AGI: ~2030
• Louis Rosenberg (Unanimous AI)
  ----> AGI: ~2030
• Ash Jafari (Ex-Nvidia-Analyst, Futurist)
  ----> AGI: ~2030
• Tony Czarnecki (Managing Partner of Sustensis)
  ----> AGI: ~2030
• Ross Nordby (AI researcher; Lesswrong-author)
  ----> AGI: ~2030
• Ilya Sutskever (OpenAI)
  ----> AGI: ~2030-35?
• Hans Moravec (Carnegie Mellon University)
  ----> AGI: ~2030-40
• Jürgen Schmidhuber (NNAISENSE)
  ----> AGI: ~2030-47?
• Eric Schmidt (Ex-Google Chairman)
  ----> AGI: ~2031-41
• Sam Altman (OpenAI)
  ----> AGI: <2032?
• Charles Simon (CEO of Future AI)
  ----> AGI: <2032
• Anders Sandberg (Future of Humanity Institute at the University of Oxford)
  ----> AGI: ~2032?
• Matt Welsh (Ex-google engineering director)
  ----> AGI: ~2032?
• Siméon Campos (Founder CEffisciences & SaferAI)
  ----> AGI: ~2032
• Yann LeCun (Meta)
  ----> AGI: ~2032-37
• Chamath Palihapitiya (CEO of Social Capital)
  ----> AGI: ~2032-37
• Demis Hassabis (DeepMind)
  ----> AGI: ~2032-42
• Robert Miles (Youtube channel about AI Safety)
  ----> AGI: ~2032-42
• OpenAi
  ----> AGI: <2035
• Jie Tang (Prof. at Tsinghua University, Wu-Dao 2 Leader)
  ----> AGI: ~2035

> live out our lives and then die

Why in this scenario where ASI is benevolent and doing research, would we not be immortal?

> Without consciousness, no understanding

You don't need consciousness nor philosophical understanding to do logic. We've already proven that with our current NNs - Just ask GPT3.5 to find bugs in your code, and that's nowhere near the limit of what NNs will be. Logic and reasoning are not as complex of tasks as we thought.

...I've never actually subbed to this place. Finding out it's anti-capitalist? Well, now I have to be subbed.