Disagree. The above statement from the blog is true.
When you have a large class imbalance skewed negative, the FPR is not very informative because it is not sensitive enough to false positives.
The definition of FPR is FP/(FP+TN). When TN is massive because of class imbalance, your model can be predicting many false positives and the FPR can still be tiny, giving you an overly rosy view of your performance and roc curves/auc that look great, when in reality your model is over predicting the positive class like crazy.
Precision doesn’t have this problem, and so PR is better.
I have worked on real applications where this has come in to play and made a huge difference because in these class imbalanced problems, the positive class is usually what you’re looking for. So if you use roc for model selection you end up flooding your predictions with FPs and it noises up the application significantly.
just__uncreative t1_it713u8 wrote
Reply to [D] Accurate blogs on machine learning? by likeamanyfacedgod
Disagree. The above statement from the blog is true.
When you have a large class imbalance skewed negative, the FPR is not very informative because it is not sensitive enough to false positives.
The definition of FPR is FP/(FP+TN). When TN is massive because of class imbalance, your model can be predicting many false positives and the FPR can still be tiny, giving you an overly rosy view of your performance and roc curves/auc that look great, when in reality your model is over predicting the positive class like crazy.
Precision doesn’t have this problem, and so PR is better.
I have worked on real applications where this has come in to play and made a huge difference because in these class imbalanced problems, the positive class is usually what you’re looking for. So if you use roc for model selection you end up flooding your predictions with FPs and it noises up the application significantly.