Viewing a single comment thread. View all comments

new_name_who_dis_ t1_iz2b35v wrote

Reply to comment by modeless in [R] The Forward-Forward Algorithm: Some Preliminary Investigations [Geoffrey Hinton] by shitboots

Is this actually biologically plausible? Seems that the idea of negative data is pretty constructed.

I see that Hinton claims it's biologically more plausible, but I don't see any justification for that statement apart from comparing it to other biologically plausible approaches, and more so spending time discussing why backprop is definitely not biologically plausible.

I'm not a neuroscientist so don't have much background on this.

25

modeless t1_iz2bm8r wrote

Well no one knows exactly what the brain is up to in there, but we don't see enough backwards connections or activation storage to make backprop plausible, so this is a way of learning without backwards connections, and that alone makes it more biologically plausible.

26

new_name_who_dis_ t1_iz2c6t0 wrote

I’ve heard that hebbian learning is how brains learn and this doesn’t seem like hebbian learning.

However idk if hebbian learning is even how neuroscientists think we learn in contemporary research

5

whymauri t1_iz38qtl wrote

As of 2019, it is what I was taught in a graduate course on associative memory and emergent dynamics in the brain. We read Hertz's Theory Of Neural Computation. This was right before people worked on Hopfield-Self Attention.

7

fortunum t1_iz2v4li wrote

Check out E-prop for recurrent spiking NN

3

Commyende t1_iz2euh0 wrote

Synapses can be excitatory or inhibitory, so that's basically like positive/negative, but I don't really know if that tracks with this algorithm 100%

8

jms4607 t1_iz38c09 wrote

I think the pos/neg here is more like contrastive learning.

8

new_name_who_dis_ t1_iz2fjjk wrote

It's negative data. It's basically contrastive learning, except without backprop. Like you pass a positive example and then a negative example in each forward pass, and update the weights based on how they fired in each pass.

It's a really cool idea, I'm just interested if it's actually biologically plausible.

I might be wrong but inhibitory synaptic connections sounds like a neural connection with weight 0, i.e. it doesn't fire with the other neuron.

6

Commyende t1_iz2wzk0 wrote

Inhibitory synapses reduce the likelihood of the downstream neuron firing.

6