1a1b

1a1b t1_jedve2g wrote

The more you talk to your dog, the more it learns about language. GPT-4 does not even have a concept of a word, just character substrings (tokens) that don't correspond to words. Similarly, your dog doesn't know the sounds we are making are "words". No such thing as words exists to dogs or GPT-4.

Despite this, with enough training data being listened to by a dog, they no longer are responding to sound like an oscilloscope. Instead they respond to patterns which represent to us the underlying meaning of speech, read emotions and context.

Similarly, GPT with enough training has begun to associate tokens with not just other tokens, but patterns of tokens. Like a dog, the more data they are trained on, the better they become at identifying these patterns and making accurate predictions about what should come next.

3