Submitted by Impressive-Injury-91 t3_1118hkt in singularity
SoylentRox t1_j8efx92 wrote
Reply to comment by FusionRocketsPlease in Anthropic's Jack Clark on AI progress by Impressive-Injury-91
Many algorithms don't show a benefit unless used at large scales. Maybe "discover" is the wrong word, if your ml researcher pool has 10,000 ideas but only 3 are good, you need a lot of compute to benchmark all the ideas to find the good ones. A LOT of compute.
Arguably you "knew" about the 3 good ideas years ago but couldn't distinguish them from the rest. So no, you really didn't know.
Also transformers are a recent discovery (2017), it required compute and software frameworks to support complex nn graphs to even develop the idea.
Viewing a single comment thread. View all comments