Viewing a single comment thread. View all comments

IntelArtiGen t1_j2x7kg9 wrote

From a very theoretical point of view, we can imagine a knowledge "A" useful for a task A, a knowledge B useful for a task B. That's how humans would apply their knowledge. But we could teach a model to learn knowledge A (or A+B) and apply it to task B, and it would eventually perform better.

Humans don't have all the knowledge and don't apply everything they could know to every tasks perfectly. Models also aren't perfect but they could do more combinations and perform better on certain tasks because of that.

But I can take another exemple. Here is a task: "a human says N images contain dogs and M images contain cats, the model must reproduce this behavior". Would a perfect model designed to exactly reproduce the human be able to outperform a human on this task? No. The human would make mistakes, and the model would reproduce these mistakes. But we don't design or train our models to exactly reproduce what a human did, that would be a risk of overfitting, we use regularizations so that even by reproducing what humans did a model can do better and not reproduce some mistakes.

7

groman434 OP t1_j2x82ze wrote

>But we don't design or train our models to exactly reproduce what a human did, that would be a risk of overfitting, so even by reproducing humans a model can do better and not reproduce some mistakes.

Can you please elaborate on this? Let's say your train data contains 10% of errors. Can you train a model that it would be more than 90% accurate? If yes, why?

Edit: My guess would be that the model during the training phase, can "find out" what are features typical for cats provided that the training set is "good enough". So even if the set contains some errors, they will not impact significantly a prediction the model can give.

1

IntelArtiGen t1_j2xa49x wrote

I can give another example. Input / Output: 1.7/0, 2/0, 2.2/1 ,3.5/0 ,4/0 ,5/0 ,8/0 ,9.6/0 ,11/1, 13/1, 14/1, 16/1, 18/1, 20/1. There is an error in this dataset: 2.2/1. But you can train a model on this set to predict 2.2/0 (a small / regularized model would do that) . You could also train a model to predict 1 for 2.2, but it would probably be overfitting. The same idea applies to any concept in input and any concept in output.

5