On discriminative joint density modeling

Abstract

We study discriminative joint density models, that is, generative models for the joint density p(c, x) learned by maximizing a discriminative cost function, the conditional likelihood. We use the framework to derive generative models for generalized linear models, including logistic regression, linear discriminant analysis, and discriminative mixture of unigrams. The benefits of deriving the discriminative models from joint density models are that it is easy to extend the models and interpret the results, and missing data can be treated using justified standard methods. © Springer-Verlag Berlin Heidelberg 2005.