Submitted by lambolifeofficial t3_zx68t9 in singularity
blueSGL t1_j21ay8x wrote
Reply to comment by treedmt in ChatGPT Could End Open Research in Deep Learning, Says Ex-Google Employee by lambolifeofficial
LLMs where it's a statistical likelihood for next token prediction benefit from more data.
That along with the truism
"You always find things in the last place you look"
can be very powerful tools.
There will be some correlation between search term and result otherwise search would be pointless. That on a large enough scale can sift signal from noise, not only in terms of search results but in delta between individual search terms.
treedmt t1_j28o94z wrote
Surely there’s some trade off between qualitative vs quantitative data?
Eg. 50 billion high quality QA pairs may beat 500B random google queries as training data.
Viewing a single comment thread. View all comments