Viewing a single comment thread. View all comments

Foundation12a OP t1_j24yu8m wrote

Reply to comment by summertime_taco in For those of you who expect AI progress to slow next year: by Foundation12a

Great rebuttle.

GATO, DALLE 2, CHAT GPT, Minerva, Pathways, etc.

https://twitter.com/jackclarkSF/status/1542723429580689408?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1542729464714391552%7Ctwgr%5E%7Ctwcon%5Es2_&ref_url=

This was all from the middle of the year.

10

manOnPavementWaving t1_j25doir wrote

Building on YEARS of ideas. They were cool, but without transformers they wouldn't exist. Without infrastructure code, they wouldn't exist. Without years of hardware improvements, they wouldn't exist. Without the ideas of normalization and skip connections, they wouldn't exist. Etc. (and this isn't even including all the alleys that were chased down, to find out they didn't work. Which isn't as clear, but definitely contributes to research).

GATO didn't even have that much to show for it, the long hoped-for skill transfer was not really there. DALLE 2 builds on CLIP and diffusion, ChatGPT builds on GPT3 and years of RL research.

You're saying something along the lines of "x is better than what came before, so the step to x is bigger than the sum of all the steps before that" and that is the worst take i've ever heard. It's definitely not how research works.

And goddamn it I'm getting deja vu cuz this bad take has been said before on this subreddit.

This rebuttal better? I'd be happy to go and list essential moments in AI in the past decade if it isn't.

7