QBoard » Artificial Intelligence & ML » AI and ML - Conceptual » Why overfit is not good?

Why overfit is not good?

  • Why do we avoid over fit?
      August 20, 2021 12:41 AM IST
    0
  • Because noise or randomnly generated fluctuations are learned which will make the model to fail in new input
      August 21, 2021 12:51 AM IST
    0
  • Overfitting is empirically bad. Suppose you have a data set which you split in two, test and training. An overfitted model is one that performs much worse on the test dataset than on training dataset. It is often observed that models like that also in general perform worse on additional (new) test datasets than models which are not overfitted.

    One way to understand that intuitively is that a model may use some relevant parts of the data (signal) and some irrelevant parts (noise). An overfitted model uses more of the noise, which increases its performance in the case of known noise (training data) and decreases its performance in the case of novel noise (test data). The difference in performance between training and test data indicates how much noise the model picks up; and picking up noise directly translates into worse performance on test data (including future data).

    Summary: overfitting is bad by definition, this has not much to do with either complexity or ability to generalize, but rather has to do with mistaking noise for signal.

    P.S. On the "ability to generalize" part of the question, it is very possible to have a model which has inherently limited ability to generalize due to the structure of the model (for example linear SVM, ...) but is still prone to overfitting. In a sense overfitting is just one way that generalization may fail.

      August 21, 2021 11:50 AM IST
    0
  • Overfitting refers to a model that models the training data too well. ... This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. The problem is that these concepts do not apply to new data and negatively impact the models ability to generalize
      August 23, 2021 7:57 PM IST
    0