Topic tracking in news streams using latent factor models

Abstract

The increasing number of published news articles and messages in social media make it hard for users to find the relevant information and to track interesting topics. Relevant news is hidden in a haystack of irrelevant data. Text-mining techniques have been developed to extract implicit, hidden information. These techniques analyze big datasets and compute “latent” features based on implicit correlations between documents and events. In this paper we develop a system that applies latent factor models on data streams. Our method allows us detecting the dominant topics and tracking the changes in the relevant topics. In addition, we explain how the extracted knowledge is used for computing recommendations based on trending topics and terms. We evaluate our system on a stream of news messages published on the micro-blogging service TWITTER. The evaluation shows that our system efficiently extracts topics and provides valuable insights into the continuously changing news stream helping users quickly identifying the most relevant information as well as current trends.