Viewing a single comment thread. View all comments

caedin8 t1_ir0z49w wrote

hyperparameter tuning should be a last step, not really necessary for 99% of production workloads, and really only for getting results publishable for papers.

I'd avoid it if possible and just go with reasonable hyperparameters. If you reach a breaking point where you can't get any better without tuning, then decide if you are trying to publish and need more accuracy, then bite the bullet and wait to publish until you finish the search, or if it is a business case, try to determine if the extra revenue from extra accuracy could offset the cost of extra compute.

3

caedin8 t1_ir0zmpw wrote

I'll add the value of machine learning is the dynamic nature of the solution. In a production situation most likely, retraining quickly with weaker hyperparameters every day would lead to a higher total applied accuracy than retraining once a month with hyperparam tuning. IF the hyperparam solution is actually better, then the problem space is very static, and you might want to rethink your ML approach

4