Submitted by to4life4 t3_11zzgzc in MachineLearning
Llukas88 t1_jdilkns wrote
There are Alpaca finetuned versions of Bloom or BloomZ on huggingface, maybe try those. Another option would be the Chat version of GPTNeoX from OpenChatKit. Both should be Open Source and free to use.
to4life4 OP t1_jdim3ip wrote
Someone said that Alpaca isn't open source though?
Llukas88 t1_jdimn2w wrote
The Alpaca model based on LLaMa isnt. The dataset, which is also called Alpaca is. If you train Bloom, which uses a permissive license, on this dataset, the Bloom license is applied to your finetuned model and you should be able to use it commercially.
to4life4 OP t1_jdin0kc wrote
Ah ok cool gotcha. Any benchmarks on Bloom performance vs Alpaca and others?
Llukas88 t1_jdiohwe wrote
Not any i know of, played around today with Alphacoom (https://huggingface.co/mrm8488/Alpacoom) and got pretty Bad results then tried a BloomZ version (https://huggingface.co/mrm8488/bloomz-7b1-mt-ft-alpaca) and got results similar to the Alpaca-Native model. Maybe read the BloomZ paper it should be a pretty good basis to build a chat model, rest should depend on your Training approach and Data.
Viewing a single comment thread. View all comments