Entity resolution in NoSQL data warehouse

Abstract

With the development of the Internet and cloud computing, there is the need of databases that will be able to store and process big data, and Not only SQL ’NoSQL’ databases are becoming increasingly used in the big data domains and have some interesting strengths such as scalability and flexibility. This paper explains the growing interest of implementing NoSQL in Data Warehouses. In addition, this paper investigates the use of data cleaning (the process of detecting and correcting or removing inaccurate records from a database) in NoSQL databases. More precisely, we are interested in adapting the data deduplication algorithms in two NoSQL models: document-oriented and column-oriented. Finally, a comparison between the implemented algorithms and the results of our simulations.