RoyalCities t1_jd0m4vt wrote on March 20, 2023 at 11:36 PM

Thanks. So bit confused here. Ot mentions needing an A100 to train. Am I able to run this off a 3090?

Bloaf t1_jd0qy7z wrote on March 21, 2023 at 12:10 AM

You can run it on your CPU. My old i7 6700k spits out 13B words a little slower than I could read them out loud. I'll test the 30B tonight on my 5600X

The_frozen_one t1_jd0sqd7 wrote on March 21, 2023 at 12:23 AM

You can run llama-30B on a CPU using llama.cpp, it's just slow. The alpaca models I've seen are the same size as the llama model they are trained on, so I would expect running the alpaca-30B models will be possible on any system capable of running llama-30B.

mycall t1_jd0ytah wrote on March 21, 2023 at 1:07 AM

alpaca-30B > llama-30B ?

The_frozen_one t1_jd125zf wrote on March 21, 2023 at 1:31 AM

Not sure I understand. Is it better? Depends on what you're trying to do. I can say that alpaca-7B and alpaca-13B operate as better and more consistent chatbots than llama-7B and llama-13B. That's what standard alpaca has been fine-tuned to do.

Is it bigger? No, alpaca-7B and 13B are the same size as llama-7B and 13B.