3deal t1_jbz6b91 wrote on March 12, 2023 at 9:20 PM

Wait, the https://huggingface.co/decapoda-research/llama-13b-hf-int4/resolve/main/llama-13b-4bit.pt is the Facebook one ?

Is it fully open now ?

Amazing_Painter_7692 OP t1_jbz7hta wrote on March 12, 2023 at 9:28 PM

It's the HuggingFace transformers module version of the weights from Meta/Facebook Research.

https://github.com/huggingface/transformers/pull/21955

MorallyDeplorable t1_jc0tuwg wrote on March 13, 2023 at 5:19 AM

It got leaked, not officially released. I have 30B 4 bit running here.

Necessary_Ad_9800 t1_jc1j36g wrote on March 13, 2023 at 11:04 AM

Where can I see stuff generated from this model?

MorallyDeplorable t1_jc1umt7 wrote on March 13, 2023 at 1:03 PM

I'm not actually sure. I've just been chatting with people in an unrelated Discord's off topic channel about it.

I'd post some of what I've got from it but I have no idea what I'm doing with it and don't think what I'm getting would be decently representative of what it can actually do.

3deal t1_jc32dgv wrote on March 13, 2023 at 6:05 PM

Does it run on a RTX 3090 ?

MorallyDeplorable t1_jc32jfw wrote on March 13, 2023 at 6:06 PM

It should, yea. I'm running it on a 4090 which has the same amount of VRAM. It takes about 20-21 GB of RAM.

3deal t1_jc32o55 wrote on March 13, 2023 at 6:06 PM

Cool, it is sad here is no download link to try it 🙂