Heteroscedastic Discriminant Analysis Using R

Abstract

For purposes of dimensionality reduction in classification Linear Discriminant Analysis (LDA) is probably the most common approach. In fact, LDA is a linear dimension reduction technique that also returns a classification rule. In the case of heteroscedasticity of the classes, Quadratic Discriminant Analysis (QDA) can be used to determine an appropriate classification rule, but QDA does not serve for dimensionality reduction. Sliced Average Variance Estimation (SAVE) has been shown to be adequate in such situations as implemented in R in the package dr. This paper presents an alternative approach for linear dimensionality reduction for situations of heteroscedastic intraclass covariances, namely Heteroscedastic Discriminant Analysis (HDA) as well as its R implementation. Furthermore, tests are suggested in order to determine the dimension for the discriminative data subspace and a generalization of HDA by regularization of the covariance matrix estimates is proposed. Examples for application of HDA in R are demonstrated as well as a small simulation study turning out that HDA is preferable to SAVE in a situation where the classes differ in both means and covariances.