Hadoop storage big data layer: Meta-modeling of key concepts and features

Abstract

In the era of information, humanity produces huge quantities of data measured in terms of terabytes or petabytes that is yet growing exponentially with time. This situation led to the emergence of a large number of big data systems and technologies that share similar architectures but with different implementations. The common architecture is composed of Data sources, Ingestion, Visualization, Hadoop Platform management, Hadoop Storage, Hadoop Infrastructure, Security, and Monitoring Layers. In our way for a unified abstract implementation, we proposed in a previous work a meta-model for data sources and ingestion layers. We relied on our previous comparatives studies to define key concepts of storage in Big Data to propose a meta-model for storage layer. Thus, in this paper, we are going to present our meta-model for storage layer. The main goal of this universal meta-modeling is to enable Big Data distribution providers to offer standard and unified solutions for a Big Data system.