Viewing a single comment thread. View all comments

ThirdMover t1_jds1kid wrote

Reply to comment by rya794 in [P] Using ChatGPT plugins with LLaMA by balthierwings

The lead may not always be obvious and the trade off from transparency may be worth it. LLMs (or rather "foundation models") will continue to capture more and more areas of competence. If I want one that - for example - forms the front end chat bot to a store I have so that people can ask for product explanations, do I need then the 500 IQ GPT-7 that won two Nobel prizes last year?

I think it's most likely that there will always be black box huge models that form the peak of what is possible with machine intelligence but what people use and interact with in practice will simply be "good enough" smaller and open source models.

20

Dwanyelle t1_jds42hs wrote

Exactly. It's not "what's the most impressive model possible?". It's "what's the most impressive model possible that can run on $1000 or less of hardware?"

15

rya794 t1_jdsev38 wrote

Yea, I agree with this, but I still don’t see what advantage the state of the art providers receive by adhering to an open protocol. If anything doing so would (on the margin) push users towards open source models when they might have been willing to pay for a more advanced model just to access certain plugins.

That being said, I do think that a standardized approach to a plugin ecosystem will arise. I just think it’s silly to expect any of the foundation model providers to participate.

4

alexmin93 t1_jduoxj4 wrote

The problem is not the model but the training dataset. That's the thing that costs millions for OpenAI. Alpacca is rather poorly performing mostly due to the fact its trained on gtp 3 generated texts

1