HateRedditCantQuitit t1_j1v0fto wrote
Reply to comment by derpderp3200 in [D] Has any research been done to counteract the fact that each training datapoint "pulls the model in a different direction", partly undoing learning until shared features emerge? by derpderp3200
The whole SGD & optimizer field is kinda this. Think about how momentum and the problem you’re talking about interact, for a small example.
Viewing a single comment thread. View all comments