A model for addressing quality issues in big data

Abstract

Big Data (BD) is everywhere and quite a lot of benefits have been derived from its usage by different organizations. Notwithstanding, there are still numerous technical and research challenges that must be tackled to comprehend and gain its full potential. The major challenges of BD are not just its processing, storage and analytics, there are also challenges associated with it that run across the BD value chain such as the data collection phase, integration and the enforcement of quality. This paper propose a DQ transformation model to evaluate BD quality from the data collection phase through to the visualization phase involving both data-driven and process-driven quality evaluation by assessing the quality of data itself first then assessing the process quality. This is still an ongoing research and hopefully will be experimented using specific Data Quality Dimensions (DQDs) like completeness, consistency, accuracy and timeliness with process quality dimensions such as Throughput, response time, latency with their corresponding metrics.