Shrinkage Approaches for Ridge-Type Estimators Under Multicollinearity

Abstract

Multicollinearity is a common issue in regression analyses that occurs when some predictor variables are highly correlated, leading to unstable least squares estimates of model parameters. Various estimation strategies have been proposed to address this problem. In this study, we enhanced a ridge-type estimator by incorporating pretest and shrinkage techniques. We conducted an analytical comparison to evaluate the performance of the proposed estimators in terms of their bias, quadratic risk, and numerical performance using both simulated and real data. Additionally, we assessed several penalization methods and three machine learning algorithms to facilitate a comprehensive comparison. Our results demonstrate that the proposed estimators outperformed the standard ridge-type estimator with respect to the mean squared error of the simulated data and the mean squared prediction error of two real data applications.