Viewing a single comment thread. View all comments

Dankmemexplorer t1_jchlw3t wrote

Reply to comment by currentscurrents in [P] nanoT5 - Inspired by Jonas Geiping's Cramming and Andrej Karpathy's nanoGPT, we fill the gap of a repository for pre-training T5-style "LLMs" under a limited budget in PyTorch by korec1234

man its funny that 250M is a toy now

how far we've come...

50

currentscurrents t1_jchn22q wrote

Computers have gotten literally 100 million times faster within my lifetime. I'm not even that old!

23

saintshing t1_jcjc3zs wrote

stolen from vitalik

>70 years is the time between the first computer and modern smart watches.

>70 years is more than the time between the first heavier-than-air flight and landing on the moon.

>70 years is 1.5x the time between the invention of public key cryptorgaphy and modern general-purpose ZK-SNARKs.

2