Viewing a single comment thread. View all comments

SodomizedPanda t1_j9jyhem wrote

And somehow, the best answer is at the bottom of the thread..

A small addition : Recent research suggests that the implicit bias in DNN that helps generalization does not only lie in the structure of the network but in the learning algorithm as well (Adam, SGD, ...). https://francisbach.com/rethinking-sgd-noise/ https://francisbach.com/implicit-bias-sgd/

27

red75prime t1_j9k0i84 wrote

Does in-context learning suggest that inductive biases could also be extracted from training data?

11