The paper you link talks about the advantages of predictive coding coming from hardware architectures that colocate compute and memory in many small, somewhat independent units. Erlang will not give you that. The BEAM VM uses 1 thread per core, limiting its parallelism to the number of cpus, and even in that context it is designed for concurrency (allowing many tasks to make progress on one thread) which is in tension with data locality to the processor. In contrast, modern backprop implementations may have limitatiins on their parallelism compared to ideal state predictive coding, but they do heavily rely on gpus for much greater parallelism than cpus can allow.
Predictive coding looks very interesting, but to be useful it needs fundamentally different hardware than commodity computers today, not just a language with good parallel semantics.
simonthefoxsays t1_iw6wznc wrote
Reply to [Project] Erlang based framework to replace backprop using predictive coding by abhitopia
The paper you link talks about the advantages of predictive coding coming from hardware architectures that colocate compute and memory in many small, somewhat independent units. Erlang will not give you that. The BEAM VM uses 1 thread per core, limiting its parallelism to the number of cpus, and even in that context it is designed for concurrency (allowing many tasks to make progress on one thread) which is in tension with data locality to the processor. In contrast, modern backprop implementations may have limitatiins on their parallelism compared to ideal state predictive coding, but they do heavily rely on gpus for much greater parallelism than cpus can allow.
Predictive coding looks very interesting, but to be useful it needs fundamentally different hardware than commodity computers today, not just a language with good parallel semantics.