[D] Models trained on academic papers?

Submitted by dewijones92 t3_zlhiog in MachineLearning

Excuse my naivety on this subject

Wouldn't it be awesome if models like OpenAI's GPT were additionally trained on all Scientific papers. Downloadable from Sci-Hub

The AI's insight would be amazing.

Is there any plan to do this?

Thoughts?

5

Comments

You must log in or register to comment.

plocco-tocco t1_j05dzmo wrote

It already is trained on that. For a version only trained on scientific papers check Galactica.

16

step21 t1_j05e2vh wrote

Isn’t this more or less what the Facebook model that was pulled did? Likely gpt is doing sth similar, at least parts of the data used are public, so you can check there

4

dewijones92 OP t1_j07em0i wrote

why was this removed? thanks

1