Viewing a single comment thread. View all comments

not_particulary t1_jd51f0h wrote

There's a lot coming up. I'm looking into it right now, here's a tutorial I found:

https://medium.com/@martin-thissen/llama-alpaca-chatgpt-on-your-local-computer-tutorial-17adda704c23

​

Here's something unique, where a smaller LLM outperforms GPT-3.5 on specific tasks. It's multimodal and based on T5, which is much more runnable on consumer hardware.

https://arxiv.org/abs/2302.00923

28

DB6135 t1_jd68vdi wrote

What are the recommended parameters? I tried the 7B model with default settings but it kept generating repeated garbage output.

9

Qwillbehr OP t1_jd6baxv wrote

I played with it for a few minutes and noticed that the 16B alpaca model gave significantly better responses. From what I can tell though the issue seems to be in how dalai prompts alpaca.CPP (just tells it to complete the sentence with all possible outputs rather then just one of the possible answers). The 16B model fixed most of it for me

4