Viewing a single comment thread. View all comments

Yomiel94 t1_jcj6i7w wrote

Reply to comment by Intrepid_Meringue_93 in Those who know... by Destiny_Knight

That’s not the whole story. Facebook trained the model, their data was leaked, and the Stanford guys fine-tuned it to make it function more like ChatGPT. Fine-tuning is easy.

40

CypherLH t1_jcjakya wrote

All You Need Is Fine-Tuning

18

vegita1022 t1_jcks65e wrote

Imagine where you'll be two more papers down the line!

12

[deleted] t1_jcob97a wrote

I hope so that it will be happen means 16GB ram and cpu or consumer gpu 😍

2

CellWithoutCulture t1_jcjku3z wrote

The specific type of fine-tuning was called Knowledge Distillation, I believe. ChatGPT taught LLaMA to chat, "stealing" OpenAI's business edge in the process.

10

visarga t1_jcjornh wrote

Everyone does it, they all exfiltrate valuable data from OpenAI. You can use it directly, like Alpaca, or for pre-labelling, or for mislabeled example detection.

They train code models by asking GPT3 to explain code snippets, then training a model the other way around to generate code from description. This data can be used to fine-tune a code model for your specific domain of interest.

15