Multivariate Regression

Abstract

In multivariate linear regression, we consider a model containingseveral y's (dependent variables) and several x's (independent variables).This is an extension of multiple regression, in which one y is regressedon several x's. A substantial review of the multiple regression modelis given before proceeding with the multivariate regression model.The review includes the model and assumptions, centered form of themodel, least squares estimation of regression coefficients in vectorform and in covariance form, estimation of the basic variance, hypothesistests of regression coefficients, R2, and selecting a subset of thex's. Subset selection can be based on all possible subsets (usingthe three criteria R2p, s2p, and Cp) or based on stepwise selection(using a partial F).In the multivariate regression model, severaly's and several x's are measured on each experimental unit. The assumptionsabout the model and about the distribution of the y's are importantand should be checked. The matrix of regression coefficients canbe estimated by least squares and can also be expressed in covarianceform. The estimates have some optimal properties if the assumptionshold.An overall regression test and a test on a subset of the regressioncoefficients are obtained in terms of Wilks' Λ, Roy's θ, Pillai'sV(s), and the Lawley-Hotelling statistic U(s). Several measures ofassociation between the y's and the x's are reviewed.Subset selectionprocedures are discussed for both the y's and the x's. These proceduresare based on stepwise selection (using a partial Wilks' Λ or thecorresponding partial F) or on all possible subsets (using threecriteria that represent matrix extensions of R2p, s2p, and Cp).Examplesare provided using real data. The problems ask for derivations andalso illustrate most procedures with real data.