Submitted by New_Yak1645 t3_11nhl03 in MachineLearning
Most AI is impossible to train(like chat GPT)
Dose LLaMa can be trained?
Although the dataset is very hard to get, It would be nice if LLaMa can be trained.
When searching for reddit, this topic cannot be searched, so I hope it becomes a discuss about HW or availability.
Thank you.
I would start searching on huggingface.co
Training from scratch required 2048 A100 for 21 days. And it seems only to be the final run.
I guess you can start to fine-tune it with much lower resources, 16 A100 seems reasonable as going lower will require quantization or partial loadings for the model.
[deleted]
Wow, they cost $15k a piece. So that's $30 million just for the GPUs! But since you only need them for 21 days, can probably sell them later on craigslist
you really think Meta spent 30 million on GPUs and then sold them on craigslist?
You can rent some (but not thousands) on vast.ai around $1.5 an hour
You're right they have FB Marketplace why would they use CL?
Good point
Any idea for finetuning llama on multi-gpu setup?
Would it be possible to train a quantzised model?
Yes, doable on a low budget if you have no fear of legal actions...
Thank you!
> if you have no fear of legal actions...
Legal actions? They can direct that to my Legal-LLaMA. ;-)
CKtalon t1_jbnccl7 wrote
If you have a few thousand A100s, sure? The dataset is fairly easily obtainable.
The next difficulty is the technical knowhow to train such LLMs.