Viewing a single comment thread. View all comments

blueSGL t1_j9qru8n wrote

I want to know how many peoples timelines predicted ChatGPT or Dalle2 or Alphafold happening when they did.

Otherwise it's just the classic "predict a game changer is going to happen once I've retired"

104

HelloGoodbyeFriend t1_j9ragu5 wrote

Would be amazed if anyone accurately predicted a LLM chatbot getting to 100m users 2 months after launch in 2022.

46

blueSGL t1_j9rf2n4 wrote

Now come on, be fair. You know that's not the point I'm making at all.

It's people working in ML research being unable to accurately predict technological advancements, not user numbers.

You might find this section of an interview with Ajeya Cotra (of biological anchors for forecasting AI timelines fame)

Starts at 29.14 https://youtu.be/pJSFuFRc4eU?t=1754

Where she talks about how several benchmarks were past early last year that surveys of ML workers had a median of 2026.
Also she casts doubt on people that are working in the field but are not working on specifically forecasting AGI/TAI directly as a source for useful information.

17

genshiryoku t1_j9svy3v wrote

No the reason why the median prediction barely got down is because we still have the exact same bottleneck and issues on the path to AGI. These haven't been solved over the past 6 years. So while we have made great strides with scaling up Transformer and specifically Large Language Models that display emergent properties. The actual issue still plays behind the scenes.

The main issue and bottleneck is training data, we're rapidly running out of usable data on the internet with the biggest models already being trained on 30% of all relevant data on the internet. If rates continue like this we might run out of usable data between 2025-2027.

We know we can't use synthetic or AI generated data to train models on because of the overfitting problem that introduces. We essentially need to either find some way to generate orders of magnitude more data (Extremely hard problem if not outright impossible). Or we need to have breakthroughs in AI architecture so that the models need to be trained on fewer data (Still a hard problem and linear in nature).

The massive progress we're seeing currently is simply just scaling up models bigger and bigger and training them on more data but once the data stops flowing these models will rapidly stagnate and we will enter a new AI winter.

This is why the median prediction barely changed. We'd need to solve these fundamental bottlenecks and issues before we'll be able to achieve AGI.

Of course the outlier possibility of AGI already emerging before running out of training data over the next 2-4 years is also a slight possibility of course.

So essentially while the current progress and models are very cool and surprising they are essentially within the realm of expected growth, because no one was doubting the AI boom to slow down before the training data ran out. We're dreading 2-4 years from now when all usable internet data has essentially been exploited already.

8

sideways t1_j9swo39 wrote

Couldn't multimodal models capable of incorporating realtime non-textual (visual, auditory, kinesthetic, etc) data be a solution?

The current generation have pretty much mastered language anyway so more text seems kinda redundant anyway.

7

beders t1_j9u0ypl wrote

The current models have not mastered language at all. They don’t know grammar. They just complete text.

It’s like claiming you know Spanish because you can pronounce the words and “read” a book. You can utter the sounds correctly but you have no clue what you are reading.

−4

rekdt t1_j9ufkwd wrote

If I can respond to a question someone asked me in Spanish then I know spanish.

7

beders t1_j9ug9kl wrote

No.

https://plato.stanford.edu/entries/chinese-room/

−1

blueSGL t1_j9ukv6h wrote

I always found that silly.

What individual parts of the brain are conscious? or is it only the brain as a gestalt that is conscious ?

3

Representative_Pop_8 t1_j9vlnsa wrote

in the Chinese room it is not the operator that knows Chinese, it is the setuo of rules + operator that clearly knows Chinese. A llm spent needed to be conscious to master language

1

beders t1_j9wj8hw wrote

The operator doesn’t know Chinese. Do I need to spell out the analogy to chatGPT?

ChatGPT is great at word embeddings and completion but is an otherwise dumb algorithm. Comparing that to human’s ability to express themselves with language is useless.

I mean if you don’t get the Chinese room experiment you might think Eliza is a master of psychology.

0

Representative_Pop_8 t1_j9wsu89 wrote

you're not getting it, the operator doesn't know chinese, but the whole setup does. chatGPT clearly understands several languages, it doesn't need to be conscious to understand.

1

beders t1_ja1diap wrote

So much for mastery:

https://twitter.com/NateSilver538/status/1629159014272581634?s=20

0

Representative_Pop_8 t1_ja1f2qz wrote

it know the language it also hallucinates, but in pretty good English. humans can also invent fake stories that doent mean they don't know the language

1

beders t1_ja1g5oh wrote

It’s almost as if it would just be a text completion engine … which it is.

0

beders t1_ja1dnlz wrote

If you can’t reason about language you can’t understand it. ChatGPT is the operator.

0

Representative_Pop_8 t1_ja1eoph wrote

chatGPT can reason about language, itv is not equivalent to the operator it is v equivalent to the v while Chinese room system, which clearly understands

1

sideways t1_j9vja6e wrote

Language mastery is a function of communication and problem solving ability in that language. Understand should be judged based on results not some mysterious inferred grammar understanding.

1

beders t1_j9wq47s wrote

You can’t just measure how well or interesting a text completion engine spits out words and proclaim it has “mastery”. Frankly that is BS.

1

sideways t1_j9wtpzn wrote

Of course... that's why I would never claim that a parrot had mastered language. It may know words but it can't use language in creative, communicative, problem solving. LLMs can.

1

fangfried t1_j9snmqy wrote

There’s gonna be something after transformers in the next couple years I can feel it

6

nillouise t1_j9s98xr wrote

I ask the same problem " how to predict the order of different capabilities that AI will obtain", and nobody know how to answer it. I think obviously it show there is no method to process this problem, people's AI timeline is useless.

But nobody know the future is more funny.

4

IntrepidHorror5986 t1_j9sh7ri wrote

You are high on hopium. AGI and LLM are absolutely unrelated! You may as well ask about how many people predicted the pills for erectile dysfunction.

−9