Temporal TF-IDF-based Twitter event summarization incorporating keyword importance

Abstract

Due to the huge content of data generated by various social networking sites like Twitter, it has become very difficult to extract the informative content from the huge lumps of data. Hundreds of millions of peoples are using Twitter, generating many posts on a daily basis; therefore, it is very challenging to extract and summarize the user-generated content. Moreover, the Twitter API also provides only latest posts in a sequential order. This motivates the dire need for a new automatic event summarization system to create event summaries supporting intelligence. In this paper, we intend to summarize the Twitter posts corresponding to Twitter hashtags to find a representative Twitter post with high quality, strong relevance, clear usefulness, and ideal acceptability. We used two approaches, namely temporal TF-IDF and temporal TF-IDF with keyword importance for finding the summary of the events. Then, we evaluate and compare these approaches using a self-generated dataset of Twitter posts and show that the system will automatically generate the summary of the selected hashtag.