Viewing a single comment thread. View all comments

86BillionFireflies t1_j1qvoxk wrote

This seems like a good time to use data augmentation. Anytime you think a model is using features of the data that you don't want it to use but that happen to correlate with the desired output, you get in there and make them NOT correlated.

E.g. add / subtract some random value from each image before using it for training, and/or multiply by some random factor, so that the scaling of different images is all over the place and the model can't effectively use that as a shortcut.

3

skn133229 OP t1_j1r2xr1 wrote

I have a pretty elaborate augmentation process implemented. I will try to randomly shift the mean of each image to see if this helps. Not sure if the fact that all images average to 0 is now a problem.

1

86BillionFireflies t1_j1r4etj wrote

Well, it's possible, and it certainly sounds like there's reason to suspect some kind of normalization related gotcha.

Maybe the scale of the values, rather than the mean being zero, is the issue?Perhaps a larger or smaller SD would change the outcome? Or a different initialization, especially if using relu. You could also try normalizing the data to have mean 0.1 and SD 1, in case it's some kind of dead relu issue. I'm really spitballing there.

2

crrrr30 t1_j1t7ybg wrote

perhaps also try random gamma or something like that? contrast could be an issue.

2