This is kind of clickbait. Cool that they reproduced some of it, but 90% of that had existed since OpenAI released the source code for InstructGPT's architecture. The real limitation is data and compute, which this repo doesn't really provide... What is really required is a huge open-source RLHF dataset (like ImageNet but for human instructions)
legocuber t1_j2eyv3m wrote
Reply to An Open-Source Version of ChatGPT is Coming [News] by lambolifeofficial
This is kind of clickbait. Cool that they reproduced some of it, but 90% of that had existed since OpenAI released the source code for InstructGPT's architecture. The real limitation is data and compute, which this repo doesn't really provide... What is really required is a huge open-source RLHF dataset (like ImageNet but for human instructions)