Stable Topic Modeling with Local Density Regularization

Abstract

Topic modeling has emerged over the last decade as a pow-erful tool for analyzing large text corpora, including Web-based user-generated texts. Topic stability, however, remains a concern: topic models have a very complex optimization landscape with many local maxima, and even different runs of the same model yield very different topics. Aiming to add stability to topic modeling, we propose an approach to topic modeling based on local density regularization, where words in a local context window of a given word have higher probabilities to get the same topic as that word. We compare several models with local den-sity regularizers and show how they can improve topic stability while remaining on par with classical models in terms of quality metrics.