Semantics based intelligent search in large digital repositories using hadoop mapreduce

Abstract

Information contained in large digital repositories consisting of billions of documents represented in various formats make it difficult to retrieve the desired information. It is necessary to develop techniques that are accurate and fast enough to retrieve the desired information from hay stack of online digital repositories. On one hand, Keyword based systems and techniques have high recall and performance, however, they have low precision. On the other hand, semantics based systems have high precision and good recall, however, their performance decreases with data growth. Therefore, to improve precision and performance, we propose semantics based searching framework using Hadoop MapReduce to process the data at large scale. We apply semantic techniques to extract required information from digital documents and MapReduce programming model to apply these techniques. Application of semantic techniques using MapReduce distributed model will result in high precision and good performance of user query result.