A Software Architecture for Enabling Statistical Learning on Big Data

Abstract

Most big data analytics research is scattered across multiple disciplines such as applied statistics, machine learning, language technology or databases. Little attention has been paid to aligning big data solutions with end-user's mental models for conducting exploratory and predictive data analysis. We are particularly interested in the way domain experts perform big data analysis by applying statistics to big data with a focus on statistical learning. In this paper we compare and contrast the different views about data between the fields of statistics and computer science. We review popular analysis techniques and tools within a defined analytics stack. We then propose a model-driven architecture that uses semantic and event processing technologies to achieve a separation of concerns between expressing the mathematical model and the computational requirements. The paper also describes an implementation of the proposed architecture with a case study in funds management.