CireNeikual t1_iw5fuqx wrote on November 13, 2022 at 2:33 AM

Reply to comment by genesis05 in [Project] Erlang based framework to replace backprop using predictive coding by abhitopia

Predictive coding is a good place to start, but I think it's also important to embrace sparsity to permit computationally efficient fully online/incremental learning. As is, predictive coding is mostly just used as a drop-in replacement for backpropagation, without really providing too many additional advantages. Predictive coding by itself doesn't permit online learning.

abhitopia OP t1_iw6lxjr wrote on November 13, 2022 at 10:17 AM

I cannot agree more with you. In fact, I think a lot of CompSci techniques like LSH, BloomFilter, etc can be used with sparse distributed representations (SDRs), which besides being robust, also provide high rubustness and capacity. And this is definitely one of the goals of the project as well.

Checkout Algorithmic Speedups vis LSH and bio-inspired hashing