ChronoPsyche t1_j2zhyic wrote on January 5, 2023 at 12:45 AM

ChatGPT is cool but any AI that only has 4000 characters of memory cannot be considered AGI or anything close to it. Not to mention all its other limitations.

crumbaker t1_j300khj wrote on January 5, 2023 at 2:54 AM

Really? How many characters can you remember?

ChronoPsyche t1_j3135qr wrote on January 5, 2023 at 9:15 AM

Let me rephrase. It only has working memory but no intermediate or long term memory. Such a human would be considered brain damaged.

DungeonsAndDradis t1_j31loh3 wrote on January 5, 2023 at 12:55 PM

Just like the WaitButWhy picture, right now we're on the left side of the exponential curve, where the AI is "brain damaged" and with a tiny shift in the timeframe, we're on the right side of the exponential curve where the AI "makes Einstein look brain damaged."

ChronoPsyche t1_j31pnq4 wrote on January 5, 2023 at 1:30 PM

The curve reaches back to the agricultural revolution, so a little shift can be anywhere from years to decades. I personally think we'll get AGI by 2030. We definitely don't have it yet though. It's also not clear if LLMs are sufficient for AGI.

Az0r_ t1_j30hq5k wrote on January 5, 2023 at 5:11 AM

It is difficult to give a precise answer to this question because the number of characters that an individual can remember can vary greatly depending on a number of factors, such as their age, education, language background, and memory skills.

However, research has shown that the average person can remember between 5 and 9 items (such as words, numbers, or characters) in their short-term memory, with some studies suggesting a number as low as 4 and others as high as 15.

PeyroniesCat t1_j305vyd wrote on January 5, 2023 at 3:33 AM

I’m dumb when it comes to AI, but that’s the biggest problem I’ve seen when using it. It’s like talking with someone with dementia.

blueSGL t1_j30p4fu wrote on January 5, 2023 at 6:22 AM

> any AI that only has 4000 characters of memory cannot be considered AGI or anything close to it.

From the comments of that article: https://www.cerebras.net/press-release/cerebras-systems-enables-gpu-impossible-long-sequence-lengths-improving-accuracy-in-natural-language-processing-models/

>The proliferation of NLP has been propelled by the exceptional performance of Transformer-style networks such as BERT and GPT. However, these models are extremely computationally intensive. Even when trained on massive clusters of graphics processing units (GPUs), today these models can only process sequences up to about 2,500 tokens in length. Tokens might be words in a document, amino acids in a protein, or base pairs on a chromosome. But an eight-page document could easily exceed 8,000 words, which means that an AI model attempting to summarize a long document would lack a full understanding of the subject matter. The unique Cerebras wafer-scale architecture overcomes this fundamental limitation and enables sequences up to a heretofore impossible 50,000 tokens in length.

Would that be enough?