A knowledge-driven approach for designing data analytics platforms

Abstract

Big data analytics technologies are rapidly expanding across all industry sectors as organisations try to make analytics an integral part of their everyday decision-making. Although there are many software tools and libraries to assist analysts and software engineers in developing solutions, organisations are looking for flexible analytics platforms that can address their specific objectives and requirements. To minimise costs, such platforms also need to co-exist with existing IT infrastructures and reuse knowledge and resources already accumulated within the organisation. To address such needs, this paper proposes the Data Analytics Solution Engineering (DASE) framework—a knowledge-driven approach supported by semantic web technologies for requirements engineering, design and development of new data analytics platforms. It includes a meta-model that captures data analytics platform requirements via a Knowledge Base, a set of guidelines that organisations can follow in engineering data analytics platforms and a reference architecture that demonstrates how to use these guidelines. We evaluate the DASE framework through two case studies and demonstrate how it can facilitate knowledge-based and requirements-driven data analytics platform engineering. The resulting data analytics platforms are observed to be user friendly, easy to maintain and flexible in handling changes to requirements. This work contributes to the body of knowledge in knowledge-driven requirements engineering, and data analytics platform engineering by providing a meta-model and a reference architecture that can be tailored to different analytics application domains.