Corpus based Automatic Text Summarization System with HMM Tagger

Abstract

herapid growth of the data in the Internet hasoverloaded the user with enormous amounts of informationwhich is more difficult to access huge volumes of documents.Automatic text summarization technique is an importantactivity in the analysis ofhigh volume text documents. TextSummarization is condensing the source text into a shorterversion preserving its information content and overall meaning.In this paper a frequent term based text summarizationtechnique with HMM tagger is designed and implemented injava. The proposed system generates a summary for a giveninput document based on identification and extraction ofimportant sentences in the document.The model consists of fourstages.In first stage, the systemdecomposesthe given text intoits constituent sentences,assigning the POS (tag) for each wordin the text and storesthe result in a table. The second stageremoves the stop words, stemming the text andapplyinglemmatization.Feature term identification is done in third stage. Finally each sentence is ranked depending on feature terms.This stage reduced the amount of the sentences in the summary in order to produce a qualitative summary.