The Combination of Contextualized Topic Model and MPNet for User Feedback Topic Modeling

Abstract

In the era of big data and ubiquitous internet connectivity, user feedback data plays a crucial role in product development and improvement. However, extracting valuable insights from the vast pool of unstructured text data found in user feedback presents significant challenges. In this paper, we propose an innovative approach to tackle this challenge by combining the Contextualized Topic Model (CTM) and the Masked and Permuted Pre-training for Language Understanding (MPNet) model. Our approach aims to create a more accurate and context-aware topic model that enhances the understanding of user experiences and opinions. To achieve this, we first search for the optimal number of topics, focusing on generating distinguishable, general, and unique topics. Next, we perform hyperparameter optimization to fine-tune the model and maximize coherence metrics. The result is an exceptionally effective model that outperforms established topic modeling methods, including LSI, NMF, LDA, HDP, NeuralLDA, ProdLDA, ETM, and the default CTM, achieving the highest coherence CV score of 0.7091. In this study, the combination of CTM and MPNet has proven highly effective in the context of user feedback topic modeling. This model excels in generating coherent, distinguishable, and highly relevant user feedback topics, capturing the nuanced nature of user feedback data. The topics generated from this model include 'Music and Audio Streaming,' 'Application Performance,' 'Banking, Financial Services, and Customer Support,' 'User Experience,' 'Other Topics,' 'Application Content,' and 'Application Features.' Our contributions include a powerful tool for developers to gain deeper insights, prioritize actions, and enhance user satisfaction by incorporating feedback into future product iterations. Furthermore, we introduce a new dataset as an open-source resource for further exploration and validation of user feedback analysis techniques and general natural language processing applications. With our proposed approach, we strive to drive business success, improve user experiences, and inform data-driven decision-making processes, ultimately benefiting both developers and users alike.