I don't see why one would have to go as far as a PID controller. The relationship between linear dynamical systems and momentum-based SGD algorithms is pretty straightforward. In fact, Lyapunov function-based analysis of SGD algorithms is pretty common.
Viewing a single comment thread. View all comments