Viewing a single comment thread. View all comments

Luckbot t1_j5xkc1f wrote

Overfitting means the system learned not the pattern you want it to learn, but rather just knows it's training data completely.

If you give it 100 pics with 50 cats and let it learn wich ones are cats without any stop criteria it will overlearn that exactly those 50 pictures are cats, but not by what the pictures have in common. It will learn stuff like "oh yeah the one with the dark blue background is a cat pic"

To prevent that you use some part of your data not for training but for quality control. You feed it only 80 pics to learn from, and use 20 only to check if they are also recognized without ever being shown to it during training.

20

alexander-prince OP t1_j5xl9nz wrote

So if I gave it the 100 pics without stopping criteria and then tested it on a much larger data set, say, 1000 pics, will the detection error increase or is this not considered overfitting since it trained on a dataset and then used on a completely different set?

4

Luckbot t1_j5xlr4y wrote

It would not recognize those and that's exactly overfitting, learning ONLY it's dataset, but not the pattern within the dataset that is general and can be applied to new data.

If this happens does also depend on how complex your ML model is though (compared to the amount of input data). The simpler it is, the more resistant it is to overfitting (but also the less complex the pattern is allowed to be).

There is a scientist joke: "If you want to perfectly fit a linear regression just give it 2 datapoints". The linear regression is pretty much the simplest model, but giving it a too small dataset makes even that useless.

7

random_web_browser t1_j5xlz98 wrote

If it was overfitted like discussed before it wouldn't recognice those 1000 pictures, because it wouldn't actually know what a cat is but just know exactly the 100 pictures you first gave it. This is exactly overfitting you are fitting the data into 100 pictures and not into detecting Cats, so any new data that you give doesn't work.

That is why you take 80 pictures from the 100 and test the algorithm with the remaining 20 to make sure it detects cats and doesn't overfit into those 80 pictures

3