In this reading, I’ll be covering the following:
- types of errors in machine learning
In machine learning or deep learning, when building any model there could be possible two types of errors.
The irreducible error is the one that cannot be reduced by the performing certain operations on the data or on the model. This could be caused by various factors like because of data unavailability or improper data sources and some of unknown factors that may affect the relationships between the input features and the output values.
The two possible reducible type of errors are:
Let’s understand the concepts behind these problems by the following.
Assume that, we have built the model that predicts the price of a product. Now, the actual values (prices of the product) based on some factors are in the range of 40–60. Having the predictive model in our hand, we try to predict the prices of the product and we will be expecting the predicted values to be in the average range of 40–60.
But what if the model predicts the values that are around the range of 80–100 or more than 100 (or) What if the model predicts all the values in the near the range of 1–20 somewhere. This is one problem that needs attention in building the model.
This type of error/ problem is called as High Bias.
Suppose assuming the same situation like above and having to predict the values (prices of the product) that could be possibly around 40–60.
What if the model predicts the values somewhere starting from the range of 20–30 and up to 70–80. We could see a lot of spread among the predicted values. But, the values we are expecting are in the range of around 40–60 only. So, this is another problem that needs to pay attention.
This type of error/ problem where there the spread of the output values is large leads to High Variance.
Dealing with Bias and Variance:
The problem of bias and variance is one of the common issues that we come across in the model development.
The arise of this problem in the construction and performance of the model leads to two kind of concepts.
Having said that, with the model we are have giving the predicted values that are far more away from the actual ones called as high bias leads to the concept of underfitting.
When we say the model is “Underfitting”, it means that the model is not able to capture the right insights/ relations within the data.
The possible reasons are:
- There is large amount of data and more features making the data more complex.
- The model is too simple to capture the proper insights.
- Reduce the regularization factor.
The under fitting issue in the model can be identified when there is more error in the training which is similar to the error rate of the validation and testing as well.
In this type, the model is trying to remember the data instead of trying to learn the proper relationships. Having the model in hand that gives the predicted values with more spread out leading to high variance also eventually comes to the over-fitting problem. When we are having a model with over-fitting, it gives the predicted values at more spread irrespective of the inputs as it was not able to draw proper conclusions. So, the model performs very well in the training data and becomes very worse in the testing phase, if new data comes into picture.
The possible reasons are:
- The data is insufficient.
- The model is over-built or more complex than required.
- Increase the regularization factor.
The over-fitting issue with the model can be identified where is low error rate in the training error and more in the validation/testing phase.
The final conclusion about the tradeoff between the bias vs variance and underfitting vs overfitting is that the model should not posses high variance, bias or high variance low bias or high bias low variance. The optimal model should possesses low variance and low bias to balance the independence and un biasness in the performance.