Submitted by billjames1685 t3_youplu in MachineLearning
LordOfGalaxy t1_ivg17bi wrote
Reply to comment by billjames1685 in [D] At what tasks are models better than humans given the same amount of data? by billjames1685
I don't think we have that much compute power. A human brain has about 100 billion neurons and they can fire at about 100Hz on average at best. Each neuron has about 1000-10000 synapses. If each firing counted as one operation for every synapse, this puts the compute power at an absolute maximum of about 100 POPS (Peta Operations Per Second). A single graphics card can manage about 100 TFLOPS these days, so this is really only about a thousand graphics cards - nothing unachievable. And the human brain does a LOT more than any model we currently have. Something like a rat brain probably has less compute power than a single graphics card, and yet in many ways our models are incapable of what a rat can do. The problem is more fundamental than just "not enough compute" IMHO.
billjames1685 OP t1_ivg2iio wrote
I am basing on this blogpost: https://timdettmers.com/2015/07/27/brain-vs-deep-learning-singularity/
Written in 2015 but the author has commented recently that he still holds the same opinion.
More recent (Jan 2022): https://www.scienceabc.com/humans/the-human-brain-vs-supercomputers-which-one-wins.html#evolution-of-computers
Generally though I don’t think there is a consensus on this because there are a lot of loosely defined terms and the brain is basically impossible to simulate.
I agree that the brain is just more optimized in general than NNs, but I’m pretty sure it’s also just way more powerful as well.
The estimated computational capacity of the brain keeps increasing as we learn more about it.
LordOfGalaxy t1_ivg90on wrote
A lot of the author's estimates are on the higher side, which takes him to the ~10^21 number. Fair enough. But even then one must concede that, say, a rat brain, with 1000 times fewer neurons, should still be within reach of modern supercomputers in terms of sheer processing power.
And even the authors of both those posts note that biological brains are VERY different from ANNs, which could confer them significant advantages. That is my own view - the biological brain is just better at what it does, and our algorithms will require significant changes to match that level of efficiency. Of course, we still need significant advances at the hardware level as well (the human brain barely uses 30W and still has some 3-6 orders of magnitude more computing power than the most powerful GPUs that easily use ten times that much power), but even with such advances we may not be able to match the biological brain unless we make some more fundamental changes to our methods.
billjames1685 OP t1_ivg9nxw wrote
Oh absolutely. Our brain is absolutely insane - with 20-30 watts it’s able to possibly have more compute than supercomputers that run on several megawatts of energy. The level of efficiency it displays is just ridiculous.
DarkCeldori t1_ivgnyd2 wrote
We also have to remember the brain has very sparse activity. IIRC on the order of 2% activity. Also most of the neurons are on the cerebellum, and humans without cerebellum still have general intelligence albeit with some difficulty with precise motion. The neocortex only has about 16Billion neurons and it is here that general intelligence occurs. That brings the 100POPs down to 16POPs times 0.02% activity = 320TOPS.
LordOfGalaxy t1_ivit0x2 wrote
True, every neuron in the brain cannot possibly be firing at the same time, and much of the brain is dedicated to just keeping us alive
Viewing a single comment thread. View all comments