Viewing a single comment thread. View all comments

Tgs91 t1_ixmylf3 wrote

What are the probability outputs during training? What about cross-entropy? Is there a difference between the training cross-entropy and the testing cross-entropy (assuming you are following the same process for testing as you are for inference). Cross-entropy is what actually gets optimized in the training process, and that's what drives probabilities towards 0 and 1. You could perform poorly on cross-entropy and still potentially get a decent accuracy. If your testing cross-entropy matches training and val cross-entropy, then the probability outputs are correct and your model just isn't very confident in it's answers. What kind of regularization are you using? Are you doing label smoothing on the prediction outputs? High regularization or label smoothing could drive your predictions away from 0 and 1.

Is there a class imbalance in your data? Is 93% a significant improvement over a more basic method like KNN? It could be that your model didn't really learn much but gets good accuracy because class imbalance makes accuracy trivial.

1