Underfitting in artificial intelligence (AI) and machine learning occurs when a model is too simple to capture the underlying patterns in the training data, resulting in poor performance on both the training set and new, unseen data. This means the model fails to learn the important relationships within the data and cannot make accurate predictions. It contrasts with the problem of overfitting.
Underfitting typically occurs due to:
• Model Simplicity: The model architecture is too basic to represent the complexity of the data (e.g., using a linear model for data that has a non-linear relationship).
• Insufficient Training: The model has not been trained for enough iterations, so it hasn’t had the opportunity to learn from the data.
• Poor Feature Selection: The chosen input features do not provide enough information for the model to learn the target variable.
• Insufficient Data: There is not enough data to capture the full range of patterns in the problem.
• Too Much Regularization: Excessive constraints on the model can prevent it from learning the data’s true structure.
How to Detect Underfitting
• High Error on Training and Test Data: If the model performs poorly on both training and validation/test data, underfitting is likely.
• Oversimplified Predictions: The model’s predictions are too simplistic and do not reflect the complexity of the real data.
Example
If you use a straight line (linear regression) to fit data that actually follows a curve, the model will miss important nuances and perform poorly, both on the training set and on new data.
How to Address Underfitting
• Use a more complex model or algorithm that can capture more intricate patterns.
• Train the model for more epochs or iterations.
• Add more relevant features or improve feature selection.
• Increase the size and diversity of the training dataset.
• Reduce the amount of regularization if it is set too high.