If you want this to be included in the training corpus of future language models, please upvote it.

Why? Well, language models are trained on the pile and common crawl. How do these dataset decide what to include? They look at reddit upvotes for one.

So you can influence what language models see in their formative years. (although they might not look at this subreddit).

HF-RLHF is the name of the dataset. As far as RLHF... what they did to LLaMA is called "Knowledge Distillation" and iirc usually isn't quite as good as RLHF. It's an approximation.

decent video

Most likely they haven't had time.

They can also use SHP and HF-RLHF.... I think they will help a lot since LLaMA didn't get the privlidge of reading reddit (unliked ChatGPT)

The specific type of fine-tuning was called Knowledge Distillation, I believe. ChatGPT taught LLaMA to chat, "stealing" OpenAI's business edge in the process.

Fantastic reply, it's great to see all those concrete advances thst made it intro prod. Thanks for sharing.

I mean... why were they not doing this already? They would have to code it but it seems like low hanging fruit

> memory efficient attention. 10x-20x increase in batch size.

That seems large, which paper has that?

> Toolformer

....oh you're right it didn't. I assumed they let it use any tool which would need RL. But it seems like they had pre-labelled ways to use tools.

Thanks for pointing that out.

Seems more like an AskML question.

But RL is for situations when you can't backprop the loss. It's noisier than supervised learning. So if you can use supervised learning, then that's what you should generally use.

RL is still used, for example the recent GATO and Dreamer v3. Or used in training an LLM to use tools like in toolformer. And also OpenAI's famous RLHF, which stands for reinforcement learning with human feedback. This is what they use to make ChatGPT "aligned" although in reality it doesn't get there.

There's a novel about his: Walkaway but Cory Docterow

ok stop it now

> kiddo

Wow you must be fun at parties

This guy is being weird dismissing you for not knowing gwern.

But I will add that Gwern is awesome and a pleasure to read. You will probably find insight in reading and skimming some of his stuff.

e.g.

• recent, gwern on bing
• technical https://gwern.net/scaling-hypothesis
• fiction https://gwern.net/fiction/clippy

That's not right. He is a researcher.

You know Bostrom right? He wrote the book on the subject and credits Yud for many things. I mean what more do you want? Influential essays? Theories? They are all out there.

He's just no much of a coder from what I can see. Having released little code.

And if you don't know who Drexler is...

I know who all these people are, yet I don't know anything lol

Yeah the jargon and meta rambling is so annoying. It's like their first priority is to show off their brains, and their second priority is to align AGI. Now they are almost finished showing of their brains, so watch out AGI.

Sometime they behave in a silly fashion. Greek philosopher's had excellent logic and deduced all kinds of wrong things. These guys seem similar at times, trying to deduce everything with philosophy and mega brains. .

IMO they are at their best when it's said in short form and it's grounded by empirical data.

There is also a lesswrong podcast or two that will read out some of the longer stuff.

Maybe they are doing a raise soon. They hired Karparthy because he's good but also because his reputation will help with raising ,especially with the narrative of a critical mass of talent. I may even be true.

Yeah I love the part where he described why it matters. It shows he really understood the paper and is filtering through the noise.

then inf, then nan

I think he mean it's now super-exponential. It's rising faster than an exponential curve.

I think he mean it's now super-exponential

There's a book called "rapture of the nerds" that explain what happens to the Amish. Read it, it's a work of rare rick and morty level genius.

Is the potato chip thing something really thinks? That subreddit is awesome, thanks for sharing

What the activation function paper?

Neural architecture search is certainly helping, you see it with optimized models coming out. And I presume chip design is.

Crazy days!

Are they really saying consciousness will emerge from LLM's, rather than that intelligence may arise from them?