Viewing a single comment thread. View all comments

ReasonablyBadass t1_j20vrrq wrote

I find the data efficiency argument weird. We consider a human fully formed at around twenty years of age. Is that really all that "data efficient"?

7

elsjpq t1_j21s4eb wrote

Well you might wanna include the couple billion years of evolutionary selection as training time as well. Otherwise, there's a ton of stuff already "baked in" to the model.

10

Cheap_Meeting t1_j21q96n wrote

Yes, they are trained on a much larger amount of language data than a human sees in their lifetime.

However, I would argue that it's a worthwhile trade-off. Computers can more easily ingest a large amount of data. Humans get feedback from the environment (like their parents), can cross-reference different modalities, and have inductive biases.

2