Viewing a single comment thread. View all comments

Seankala t1_j8r2317 wrote on February 16, 2023 at 9:52 AM

Reply to comment by currentscurrents in [D] Lion , An Optimizer That Outperforms Adam - Symbolic Discovery of Optimization Algorithms by ExponentialCookie

> ...just the hyperparameter was the optimizer design itself.

Probably one of the best things I've read today lol. Reminds me of when old colleagues of mine would have lists of different PyTorch optimizers and just loop through them.