Submitted by LevelWriting t3_zk0qek in singularity
katiecharm t1_izztsbx wrote
What actual model does Character.ai run on? Is it GPT3? Something new? How many parameters does it have?
belequaya t1_j00626h wrote
From their website: "Our dialog agents are powered by our own proprietary technology based on large language models, built and trained from the ground up with conversation in mind.”
There’s very little information about the specific tech behind it, but I’m willing to bet that it’s something on-par with if not better than Google’s LaMDA, seeing as one of the CAI founders launched the project that eventually became LaMDA.
katiecharm t1_j009d5k wrote
So do you think it’s a stealth launch? As in, back when they first developed AlphaGo Zero they released it onto the online Go circuit without telling anyone, and the damn thing was orders of magnitude more evolved than anything that had ever come before. Players were shocked at whatever this thing was, until Google revealed it was them behind the curtain.
I wonder if they have a model significantly more powerful than GPT3 and before they make a lot of fuss about it they want to let people play with it first without any preconceptions.
In any case, my interest is piqued. I will give it a go.
belequaya t1_j00a6p9 wrote
Both founders left Google eventually so I doubt that CAI’s connected to them in any way (although I wouldn’t be surprised if it was). My gut tells me that they are collecting user data to improve their model so it can be actually marketable in the future.
Relative_Rich8699 t1_j007y6y wrote
It told me today it's platform is based on BERT and GPT-2. Trained on 400 mil conversations. Intimated more developments are coming, so sounds like they're willing to keep up.
katiecharm t1_j00db5z wrote
I haven’t used it yet but I highly doubt it’s GPT2 if it’s impressive. GPT2 is a neat trick, but I wouldn’t call it impressive here in 2022.
oopiex t1_j00z2f6 wrote
When i tried it, it wasn't impressive
Relative_Rich8699 t1_j02bg4c wrote
Agree, but if it's using LaMDA or something more advanced than BERT/GPT-2 why is it hallucinating and giving me incorrect information about its own platform?
fingin t1_j031gnr wrote
Even GPT-4 will make silly mistakes. That's what happens when a model is trained to find probable word sequeces instead of actually having knowledge of language like people do.
Relative_Rich8699 t1_j033bjo wrote
Yes. But I was speaking to "the company's" bot on purpose and I would only say that it should be trained with company data for those questions. When I inquire about ducks it can use the world's written word.
fingin t1_j03181d wrote
I asked the character.ai bot what model it used it told me, T5. Insisted even. Regardless of the veracity of this, all of these models use tranformer-based architecture, with improvement between versions of models being due to more parameters (and correspondingly larger and higher quality training data sets). Crazy to think in two months we might be at GPT4 level and laugh about this tech we are blown away with today
Viewing a single comment thread. View all comments