Viewing a single comment thread. View all comments

KaiSix88 t1_j1ttc7j wrote

>So he predicted the trajectory not based on statistics, but by some kind of a new rule, perhaps one that closely resembles the laws of physics.

You actually hit it right on the money. That's where the sparsity comes into play. It is technically probabilistic though.

Imagine this for example. I have 100 neurons, only 3 neurons can turn on at a time. 100 choose 3, that is 161700 combinations. We'll call the code that lights up for a ball falling code G for gravity. We'll also say that your motor cortex will fire if it has at least 2 neurons that are the same.

The odds of any other code has an exact match is 1/161700. Very unlikely that anything other than a falling object overlaps with code G. However, there are noisy partial codes, A B and C (3 choose 2), that can overlap with G.

Because these 100 neurons are representing something similar in nature in the same part of the brain, the full code variants of codes A B and C will have meaningful overlap with G because they were formed from the same inputs. This leaves us 3 * 98 full overlapping codes. 98 variants per noisy partial because 3 neurons need to be on at a time, each partial is missing 1 neuron, and there are 98 other neurons to choose from.

As you may have guessed by now, your windy variants are the other overlapping codes. You can call that set W for windy. Only the codes with overlap of 2. So now you have 295(3*98+1) codes that can activate under falling conditions.

But even then, with that many codes between the full W set and code G, 295/161700 is still less than a percent chance of probability of a random code triggering a gravity related thought. In this scenario we haven't considered temporal codes, but this is enough to illustrate how implicit probabilities can arise out of sparse distributed codes.

If you are actually interested in this field, Kanerva and locality sensitive hashes will be right up your lane.

1