Big data-driven digital ecosystem framework for railway reporting

Abstract

In our application research, we propose a Digital Ecosystem approach to overcome data integration, orchestration and quality challenges in railway reporting system using Big Data technologies. We are building a Digital Ecosystem Framework consisting of different Agents, where each Agent is an essential part of the Railway Reporting Management System. In this work, we address different problems in building digital ecosystem including integration problems, orchestration problems and data quality problems. We present a proprietary solution called the Digital Ecosystem Reporting Framework (DERF) for building robust, reliable, fault-tolerant, scalable and high-loaded data pipelines of the Railway Reporting Management System based on Big Data technologies. DERF integrates different Digital Agents such as main ETL-pipeline Agents, technical data quality Agents, business data quality Agents, BI-services integration Agents and high-level data orchestration Agent. A test implementation of DERF has been performed for Railway Reporting Management System using KPI reporting data of the real Railway company.