> the french will build it

funny here knowing who is behind BLOOM

It's pretty cool and thanks for providing the playground. I wouldn't have bothered without it. I think it's very valuable but also is quite costly, both economically and computationally, while creating privacy risks (all your data going through OpenAI) so... again in some situation I can imagine it being quite powerful but in others and absolute no. That being said there are others models (I just posted on /r/selfhosted minutes ago about the HuggingFace/Docker announcement enabling us to run Spaces locally) e.g Alpaca or SantaCoder or BLOOM that might enable us to follow the same principle, arguably with different quality, without the privacy risks. Have you considering relying on another "runtime"?

Well I just did so please explain why not, genuinely trying to learn. I'd also be curious if you have a list of trained models compared by cost. I only saw some CO2eq order of magnitude equivalent but not rough price estimations so that would help me to get a better intuition as you seem to know more about this.

That being said the point was that you don't necessarily need to train anything from scratch or buy anything to have useful results, you cant rent per hour on cloud and refine existing work, no?

$500 https://github.com/tatsu-lab/stanford_alpaca#data-generation-process

Thanks but that only clarifies from the UX side, we don't know know if OpenAI does save them and could decide to include past sessions in some form, as a context even with the current model, do we?

Does ChatGPT actually do that currently, namely keep track of your past prompts and makes a model of your tastes or values, so that "me" here is meaningful?

PS: not sure why the downvote. Is it an offensive or idiotic question?

in case you didn't follow https://www.reuters.com/technology/chinese-search-giant-baidu-introduces-ernie-bot-2023-03-16/ but nothing open source AFAICT.

Still says absolutely nothing if you don't know what a cat is.

> What we want is a model that can represent the "semantic content" or idea behind a sentence

We do but is it what embedding actually provide or rather some kind of distance between items, how they might relate or not between each other? I'm not sure that would be sufficient for most people to provide the "idea" behind a sentence, just relatedness. I'm not saying it's not useful but arguing against the semantic aspect here, at least from my understanding of that explanation.

It's not controversial as long as you don't share it and make money with it, you are pretty much free to do whatever you want.

If you plan to share the output, meaning here what's generated, not just the code and checkpoints, or a training set that's under copyright, publicly then it's another question entirely and if you are serious about that I recommend seeking legal advice.

Indeed, makes me wonder what's the share from Transformers alone or untouched Docker images.

Not for a while but I started doing it again, did this little thing for Alameda Research and it went pretty great for a while but ... yeah, maybe I shouldn't. (sarcasm, just being facetious with the latest crypto scandal)

This is not my field but I find this question genuinely surprising.

Why would one even consider this unless prototyping from the actual jungle?

In any other situation where you even just have a 3G connection then delegating to the cloud (or your own on premise machines available online behind a VPN) seems much more efficient as soon as you have any inference and even more so training to run.

Why do I feel the question itself to be surprising? Well because ML is a data based field so the question can be answered with a spreadsheet. Namely your "model" would be optimizing for faster feedback in order to learn better about your problem with as costs your hardware but also your time. If you do spend X hours tinkering with an M1 (or M2 or even "just" a 4090) versus A100 in that or a random cloud e.g AWS or a local OVH by booting on a generalist distribution like Ubuntu versus dedicated setups like lambdalabs.com or coreweave.com or even higher level like HuggingFace on their own infrastructure then IMHO that does give you some insight.

Everything else seems anecdotal because others might not have your workflow.

TL;DR: no unless they are minuscule models and of course if you use it to ssh on remote machines but IMHO you have to figure it out yourselves as we all have different needs.

PS: to clarify and not to sound like an opinionated idiot, even though it's not my field I did run and trains dozens of models locally and remotely.