I8K|DQ-BigData: I8K architecture extension for data quality in big data

Abstract

During the execution of business processes involving various organizations, Master Data is usually shared and exchanged. It is necessary to keep appropriate levels of quality in these Master Data, in order to prevent defects and failures in the business processes. A way to support the decision about the usage of data in business processes is to include information about the level of quality alongside the Master Data. ISO/TS 8000 parts 100 to 140, may support the provision of this kind of information in a usable manner. Specifically I8K, a reference implementation from academic sources of the aforementioned standard parts (ISO/TS 8000:100-140), may be used for this objective. Regrettably, I8K is not aimed to support the assessment of large Master Data volumes and does not reach the required efficiency in Big Data surroundings. This paper describe an extension of I8K to resolve those problems of efficiency in Big Data projects.