Constrained log-likelihood-based semi-supervised linear discriminant analysis

Abstract

A novel approach to semi-supervised learning for classical Fisher linear discriminant analysis is presented. It formulates the problem in terms of a constrained log-likelihood approach, where the semi-supervision comes in through the constraints. These constraints encode that the parameters in linear discriminant analysis fulfill particular relations involving label-dependent and label-independent quantities. In this way, the latter type of parameters, which can be estimated based on unlabeled data, impose constraints on the former. The former parameters are the class-conditional means and the average within-class covariance matrix, which are the parameters of interest in linear discriminant analysis. The constraints lead to a reduction in variability of the label-dependent estimates, resulting in a potential improvement of the semi-supervised linear discriminant over that of its regular supervised counterpart. We state upfront that some of the key insights in this contribution have been published previously in a workshop paper by the first author. The major contribution in this work is the basic observation that a semi-supervised linear discriminant analysis can be formulated in terms of a principled log-likelihood approach, where the previous solution employed an ad hoc procedure. With the current contribution, we move yet another step closer to a proper formulation of a semi-supervised version of this classical technique. © 2012 Springer-Verlag Berlin Heidelberg.