Viewing a single comment thread. View all comments

FusionRocketsPlease t1_j8eeegf wrote

Reply to comment by SoylentRox in Anthropic's Jack Clark on AI progress by Impressive-Injury-91

What do you mean computation is needed to discover algorithms?

5

SoylentRox t1_j8efx92 wrote

Many algorithms don't show a benefit unless used at large scales. Maybe "discover" is the wrong word, if your ml researcher pool has 10,000 ideas but only 3 are good, you need a lot of compute to benchmark all the ideas to find the good ones. A LOT of compute.

Arguably you "knew" about the 3 good ideas years ago but couldn't distinguish them from the rest. So no, you really didn't know.

Also transformers are a recent discovery (2017), it required compute and software frameworks to support complex nn graphs to even develop the idea.

7