Correlation Scaled Principal Component Regression

Abstract

Multiple Regression is a form of model for prediction purposes. With large number of predictor variables, the multiple regression becomes complex. It may underfit on higher number of dimension (variables) reduction. Most of the regression techniques are either correlation based or principal components based. The correlation based method becomes ineffective if the data contains a large amount of multicollinearity, and the principal component approach also becomes ineffective if response variables depends on variables with lesser variance. In this paper, we propose a Correlation Scaled Principal Component Regression (CSPCR) method which constructs orthogonal predictor variables having scaled by corresponding correlation with the response variable. That is, the construction of such predictors is done by multiplying the predictors with corresponding correlation with the response variable and then PCR is applied on a varying number of principal components. It allows higher reduction in the number of predictors, compared to other standard methods like Principal Component Regression (PCR) and Least Squares Regression (LSR). The computational results show that it gives a higher coefficient of determination than PCR, and simple correlation based regression (CBR).