Inference in supervised latent Dirichlet allocation

Abstract

Supervised latent Dirichlet allocation (Supervised-LDA) [1] is a probabilistic topic model that can be used for classification. One of the advantages of Supervised-LDA over unsupervised LDA is that it can potentially learn topics that are inline with the class label. The variational Bayes algorithm proposed in [1] for inference in Supervised-LDA suffers from high computational complexity. To address this issue, we develop computationally efficient inference methods for Supervised-LDA. Specifically, we present collapsed variational Bayes and MAP inference for parameter estimation in Supervised-LDA. Additionally, we present computationally efficient inference methods to determine the label of unlabeled data. We provide an empirical evaluation of the classification performance and computational complexity (training as well as classification runtime) of different inference methods for the Supervised-LDA model and a classifier based on probabilistic latent semantic analysis. © 2011 IEEE.