Viewing a single comment thread. View all comments

EverythingGoodWas t1_je612lg wrote

You aren’t going to train an LLM on company data. You could fine tune an existing one with company data, but creating an LLM from scratch is an absolutely massive compute task. If you are trying to make a closed domain question answering system, that uses your company’s data, you basically need to create a full pipeline from parsing, searching, and finally pushing the context and question to a language model.

15

visarga t1_je6kqvw wrote

I'd rather fine-tune the LLM on company documentations than feeding it through retrieval. Does anyone have experience with fine-tuning GPT-3 on a new text? Can it answer questions or freely use information from this text?

11

AlmightySnoo t1_jea6dra wrote

I'm just curious, how are you supposed to fine-tune a model on company data if the current licences (either explicitly, or implicitly through the licence of the training data) on model weights prohibit commercial use?

1

EverythingGoodWas t1_jea7gbo wrote

You wouldn’t, that would be a direct violation of that license. I would imagine they have a commercial use license as well though.

2