Ordinary Least Squares and Robust Estimators in Linear Regression: Impacts of Outliers, Error and Response Contaminations

Abstract

The Ordinary Least Squares Estimator (OLSE) is the best method for linear regression if the classical assumptions are satisfied for estimating weights. When these assumptions are violated, the robust methods give more reliable estimates while the OLSE is strongly affected adversely. In order to assess the sensitivity of some estimators using more than five criteria, a secondary dataset on Anthropometric measurements from Komfo Anokye Teaching Hospital, Kumasi-Ghana, is used. In this study, we compare the performance of the Huber Maximum Likelihood Estimator (HMLE), Least Trimmed Squares Estimator (LTSE), S Estimator (SE) and Modified Maximum Likelihood Estimator (MMLE) relative to the OLSE when the dataset has normal errors; 10, 20 and 30 percent outliers; 20% error contamination and lognormal contamination in the response variable. In the assessment, we use coefficients and their standard errors, relative efficiencies, Root Mean Square Errors, and the coefficients of determination of the estimators. We also use the power of the test to assess the effects of the aberrations on the post hoc power analysis of the estimators. The results show the SE and MMLE outperform the HMLE and LTSE while the OLSE breaks down completely. The LTSE performs well when the trimming is done to eliminate only the outliers. Also, SE and MMLE resist the effect of all aberrations in the data and also have good post hoc power analysis.