Viewing a single comment thread. View all comments

EmbarrassedHelp t1_jbjqy4o wrote

Human brains have structural components / shapes that likely help them learn languages easier:

https://en.wikipedia.org/wiki/Wernicke%27s_area https://en.wikipedia.org/wiki/Broca%27s_area

Human brains also start off with way more parameters than needed, and language is most effectively learned before the synaptic pruning reduces the number of parameters.

6

harharveryfunny t1_jbjxolz wrote

The LLM name for things like GPT-3 seems to have stuck, which IMO is a bit unfortunate since it's rather misleading. They certainly wouldn't need the amount of data they do if the goal was merely a language model, nor would we need to have progressed past smaller models like GPT-1. The "predict next word" training/feedback may not have changed, but the capabilities people are hoping to induce in these larger/ginormous models is now way beyond language and into the realms of world model, semantics and thought.

2