Viewing a single comment thread. View all comments

3deal t1_jc32dgv wrote

Reply to comment by MorallyDeplorable in [P] Discord Chatbot for LLaMA 4-bit quantized that runs 13b in <9 GiB VRAM by Amazing_Painter_7692

Does it run on a RTX 3090 ?

2

MorallyDeplorable t1_jc32jfw wrote

It should, yea. I'm running it on a 4090 which has the same amount of VRAM. It takes about 20-21 GB of RAM.

2

3deal t1_jc32o55 wrote

Cool, it is sad here is no download link to try it 🙂

1