Viewing a single comment thread. View all comments

athos45678 t1_je7ercw wrote

Train a Llama LoRa model. The 30 b model isn’t too expensive to tune (40 bucks ish), and is ridiculously capable.

You just need to format the data in a long text doc with each prompt separated by two line breaks. I found it worked best in the alpaca style, where you have a single line break after the prompt, like “write a function that sorts this table in python def sort():” followed by the written out code, and then the double line break to signal the start of the next input.

Then use the simple-llama trainer app to make it all easy.

3

netham91 t1_je7jpai wrote

Can you share more steps on this and also share some relevant links? Thanks.

1

athos45678 t1_je82thk wrote

So as far as set up goes, you just need to: “”” Git clone https://github.com/lxe/simple-llama-finetuner Cd simple-llama-finetuner Pip install -r requirements.txt Python app.py ## if you’re on a remote machine (Paperspace is my go to) then you may need to edit the last line of this script to set ‘share=True’ in the launch args “””

Then you should get a link for the gradio web app. Copy and paste the code samples in the format described before in the input text box. It will look something like this:

“”” Write a code snippet that sorts a function Def sort(arr):

  Return arr.sorted()

Some other code snippet input

Some answer

Etc. “””

Edit: I’m drinking with friends sorry i can’t format better. Single line break between prompt and observed correct response, double line break between prompt instances.

3

jaxolingo OP t1_je98vdm wrote

nice thanks i'll take a look at LoRa models

1