Evaluating thesaurus-based topic models

Abstract

In this paper, we study thesaurus-based topic models and evaluate them from the point of view of topic coherence. Thesaurus-based topic models enhance the scores of related terms found in the same text, which means that the model encourages these terms to be on the same topics. We evaluate various variants of such models. First, we carry out a manual evaluation of the obtained topics. Second, we study the possibility to use the collected manual data for evaluating new variants of thesaurus-based models, propose a method and select the best its parameters in cross-validation. Third, we apply the created evaluation method to estimate the influence of word frequencies on adding thesaurus relations for generating coherent topic models.