Efficient Learning for undirected topic models

Abstract

Replicated Softmax model, a well-known undirected topic model, is powerful in ex-tracting semantic representations of docu-ments. Traditional learning strategies such as Contrastive Divergence are very inef-ficient. This paper provides a novel esti-mator to speed up the learning based on Noise Contrastive Estimate, extended for documents of variant lengths and weighted inputs. Experiments on two benchmarks show that the new estimator achieves great learning efficiency and high accuracy on document retrieval and classification.