Viewing a single comment thread. View all comments

Puzzleheaded_Acadia1 t1_jdjvola wrote

I have questions can I fine-tune the gpt-neo-x 125m parameters on chat dataset to give me a decent answer like human because when I run it give me random characters

1

liyanjia92 OP t1_jdjwfnh wrote

It maybe better to submit an issue on github so that i can point you to some code with context. if you are talking my code, you need to convert the weights and load it into GPT class before running SFT training. otherwise there might be mismatch in weights and it could just output random stuff.

2