[D] Fine-tuning open source models on specific tasks to compete with ChatGPT?

Submitted by jaqws t3_10dljs6 in MachineLearning

As the title says, I'm curious about using open source models like GPT-J, GPT-NeoX, Bloom, or OPT to compete with ChatGPT for *specific use-cases* such as explaining what a bit of code does. ChatGPT does this task quite well, but it's closed-source nature prevents it from being useful in documenting or commenting proprietary code. There's also limitations such as the amount of text ChatGPT will read or respond with.

Getting beyond these limitations is something I'm interested in pursuing, perhaps with the help of somewhere in this subreddit. Some assumptions you can safely make:

  1. We can get (lots of) funding for the training, hardware, etc...
  2. The end product should be on-premises
  3. The inference does not actually need to run very quickly. If it costs millions to buy enough GPUs just due to VRAM limitations, we could simply run on CPUs and utilize ram, as long as inference could be done a few times per day.

So I guess my questions are where would we start? What model is best to fine-tune? How would you specifically fine-tune to improve specific use cases?

41

Comments

You must log in or register to comment.

avocadoughnut t1_j4m12v2 wrote

There's currently a project in progress called OpenAssistant. It's being organized by Yannic Kilcher and some LAION members, to my understanding. Their current goal is to develop interfaces to gather data, and then train a model using RLHF. You can find a ton of discussion in the LAION discord. There's a channel for this project.

19

LetGoAndBeReal t1_j4mihya wrote

I looked through their repo, but I'm not understanding something: what is the foundational model that they plan to use and where/how will the model be run?

4

avocadoughnut t1_j4n5sp8 wrote

From what I've heard, they want a model small enough to run on consumer hardware. I don't think that's currently possible (probably not enough knowledge capacity). But I haven't heard that a decision has been made on this end. The most important part of the project at the moment is crowdsourcing good data.

5

LetGoAndBeReal t1_j4n6rfa wrote

Wow, that seems awfully ambitious given that GPT3.5 requires something like 700GB of RAM and the apparent unlikeliness that SoTA model sizes will get smaller anytime soon. Interesting project to watch, though.

5

avocadoughnut t1_j4n8bp2 wrote

Well, there are projects like WebGPT (by OpenAI) that make use of external knowledge sources. I personally think that's the future of these models: moderated databases of documents. The knowledge is much more interpretable and modifiable that way.

5

MegavirusOfDoom t1_j4oelbd wrote

less than 500MB is used for code learning, 690GB is used for culture, geography, history, fiction and non-fiction... 2GB for cats, 2GB bread, horses, dogs, Cheese, Wine, Italy, France, Politics, Television, Music, Japan, Africa. less than 1% of the training is on science and technology, i.e. 300MB is biology, 200MB chemistry, 100MB physics, 400MB maths...

2

yahma t1_j4owot0 wrote

This may be the size of the datasets, but i it's hard to say how many parameters will be needed for a good llm that's just really good at explaining code.

5

MegavirusOfDoom t1_j4pfdi1 wrote

Then we'd have to crawl all of stack exchange, all of wiki, and 1 terabyte of programming books... This "generalist NLP" is for article writing, for poetry.

I'm a big fan of teaching ChatGPT how to interpret graphs, the origin lines, to record in a vector engine that is couple with the NLP. For a coding engine, I believe NLP should be paired with a compiler, just like a maths specialized NLP should also have a mathlab type engine.

2

Acceptable-Cress-374 t1_j4m7mee wrote

> Their current goal is to develop interfaces to gather data, and then train a model using RLHF

Potentially naive question, as I don't have much experience with LLMs. Has anyone tried using existing SotA (paid) models like davinci / gpt3 instead of RLHF? They seem to be pretty good at a bunch of focused tasks, especially in few-shot. Does that make sense?

2

avocadoughnut t1_j4mci2y wrote

ChatGPT is GPT3 + instructional finetuning + RLHF for alignment. If you're talking about using those models ro gather training data, that's against OpenAI TOS, so I've heard. The goal is to make something that isn't closed source, something you can run yourself.

9

sad_dad_is_a_mad_lad t1_j4ohl7t wrote

I don't think there are any laws that protect their data in this way, except perhaps contract law because they have a hidden ToS that you have to accept to use their service. As long as you use it for free though, I'm not sure there is consideration, and well... I don't know how they would go about proving misuse or damages.

Certainly it would not be copyright law, given that GPT3 itself was trained on copyrighted data...

2

Zondartul t1_j4mb6rm wrote

So using a big network to teach a small network? That's a thing people do. See teacher-student learning, and distillation.

1

TheTerrasque t1_j4p9kk7 wrote

There is a project called Petals that have BLOOM running for everyone to use. It's distributing the model over many machines and thus allows it to run on consumer hardware. There is a PoC chat at http://chat.petals.ml/

They just converted BLOOMZ and is currently setting up a network for that. That should be more suited for a chat interface. There's still missing gpu's though, so would be great with some more servers if people got some spare compute.

4

Ham05 t1_j4noofg wrote

If the goal is for a commercial endeavor I suggest bringing on an ML-specialized shop. Good ones can knock this out in a few sprints. PM me if you need more info.

2

theaimlguy t1_j4pceky wrote

If it was possible to try distillation on ChatGPT to produce smaller models which could run on mobile hardware, it would have been great!

2