rehrev t1_it9f0a4 wrote
Who actually tries to predict the sota level in the future, especially on language modeling?
Cheap_Meeting t1_itb39jn wrote
I was wondering the same. Here is the answer:
https://bounded-regret.ghost.io/forecasting-math-and-mmlu-in-2023
rehrev t1_itb46ky wrote
Oh expert blogs
Qumeric t1_itkg1un wrote
There are things called prediction markets and prediction aggregators. See https://www.metaculus.com/questions/11676/mmlu-sota-in-2023-2025/ for up-to-date predictions for MMLU SotA.
Also, see https://www.lesswrong.com/posts/arveXgFbJwascKtQC/forecasting-ml-benchmarks-in-2023 for a detailed prediction from a single forecaster.
Viewing a single comment thread. View all comments