Improving a New Data Lake Architecture Design Based on Data Ponds and Multi-Agent Paradigms

Abstract

For several years, Big Data has been considered the next evolution for processing various data in Data Lake. However, one of the major difficulties for Big Data development is its need for structured knowledge. This is why, a large part of research focuses on ontologies formalization that has, among other things, given rise to ad hoc languages as Web Ontology Language (OWL). In this paper, ontologies were embedded into Data Ponds to operate and cooperate with agents. Indeed, a new architecture has been built based on traditional multi-agent system in a Big Data context. This architecture enables end users to access to information from Data Lake in real time with the slightest effort. This issue has been the subject of research for nearly ten years, but it remains one of the major obstacles to Big Data development.