to4life4 OP t1_jdim3ip wrote on March 24, 2023 at 5:31 PM

Reply to comment by Llukas88 in [D] What is the best open source chatbot AI to do transfer learning on? by to4life4

Someone said that Alpaca isn't open source though?

Llukas88 t1_jdimn2w wrote on March 24, 2023 at 5:34 PM

The Alpaca model based on LLaMa isnt. The dataset, which is also called Alpaca is. If you train Bloom, which uses a permissive license, on this dataset, the Bloom license is applied to your finetuned model and you should be able to use it commercially.

to4life4 OP t1_jdin0kc wrote on March 24, 2023 at 5:37 PM

Ah ok cool gotcha. Any benchmarks on Bloom performance vs Alpaca and others?

Llukas88 t1_jdiohwe wrote on March 24, 2023 at 5:46 PM

Not any i know of, played around today with Alphacoom (https://huggingface.co/mrm8488/Alpacoom) and got pretty Bad results then tried a BloomZ version (https://huggingface.co/mrm8488/bloomz-7b1-mt-ft-alpaca) and got results similar to the Alpaca-Native model. Maybe read the BloomZ paper it should be a pretty good basis to build a chat model, rest should depend on your Training approach and Data.