Fact based search engine: News fact finder utilizing naive bayes classification

Abstract

There are a number of quality news sources available on the Internet. Searching through all these sources for facts related to a certain subject would be exhaustive for a user. We developed a niche sentence level search engine called News Fact Finder in order to provide users with factual information relevant to the query. Sentence level search is based on the intuition that if all the query words are within the same sentence, that result is more relevant than a result containing the query words in remote parts of the text.We therefore use suffix arrays which excel at exact substring matching to index our database. Our framework uses a Naive Bayes classifier for classification of sentences as facts and opinions. Ranking was performed at the document level, such that a document with many related facts would be ranked higher. News Fact Finder performs competitively on a large collection of news documents in providing relevant fact-based results to users. This is a novel approach to perform quality-based searching, ranking, indexing and categorization of news information. © Springer-Verlag Berlin Heidelberg 2013.