TemetN t1_iw08nun wrote on November 11, 2022 at 10:51 PM

Saw this elsewhere, but while nice if true (and the Gwern thing is interesting) it seems like a hype article. It would be a huge deal if they really did manage to do that to training costs though.

manOnPavementWaving t1_iw0c7h3 wrote on November 11, 2022 at 11:18 PM

Most news around GPT-4 is. The "Cerebras partnership" was always just a mention of GPT-N by Cerebras to hype up their processor, OpenAI had no say in that. (also not sure if .e6 is 100k-1M or 1M-10M). The only leak that Im sure came from Sam was "the model won't be much bigger than GPT-3 and be text only", which Id say is the only one to trust (although it can be outdated).

Phoenix5869 t1_iw1bumc wrote on November 12, 2022 at 4:23 AM

Yes its probably hype just like most of the articles on here, from what ive heard its the same as gpt 3 but its scaled down and does the same thing which is good bc u need less paramaters to do the same thing

AdditionalPizza t1_iw2ftl8 wrote on November 12, 2022 at 1:01 PM

I can't imagine it will be the same as gpt3 because even small models now outperform gpt3 by a ton.

While it might be a smaller scale model it will still most likely be way more powerful. I do hope we get a full size model with all the new techniques though, but I wouldn't mind a pocket size one either.