Towards big data in education: The case at the Open University of the Netherlands

Abstract

This chapter describes the architecture for an enterprise-level big data infrastructure at the Open University of the Netherlands. This infrastructure aims to capture dispersed data from various heterogeneous data sources into a single ledger of immutable data, which allow the development of the so-called ‘smart services’. Four major requirements for this infrastructure are derived: scalability, availability, reliability, and flexibility. These requirements together with the sheer size of the ledger lead to an architecture that is horizontally scalable by using distributed data systems. Next, the Lambda and Kappa reference architectures are discussed as they both meet the requirements. However, the authors argue that the Kappa architecture is more appealing due to a reduced complexity by using a single technology stack. Finally, the resulting architecture is outlined in detail.