Comparative study of LASSO, ridge regression, preliminary test and stein-type estimators for the sparse gaussian regression model

Abstract

This paper compares the performance characteristics of penalty estimators, namely, LASSO and ridge regression(RR), with the least squares estimator (LSE), restricted estimator (RE), preliminary test estimator (PTE) and the Stein-type estimators. Under the assumption of orthonormal design matrix ofa given regression model, we find that the RR estimator dominates the LSE, RE, PTE, Stein-type estimators and LASSO estimator uniformly, while, similar to [17], neither LASSO nor LSE, PTE and Stein-Typeestimators dominates the other. Our conclusions are based on the analysis of L2-risks and relative risk efficiencies (RRE) together with the RRE related tables and graphs.