Exploring DevOps for data analytical system with essential demands elicitation

Abstract

DevOps is an emerging concept and methodology for bridging the gap in the process of software development. At present, applying DevOps to data analytical system (DAS) is increasingly embraced. But the characteristics of this system, such as data protection, always leads to a series of constrains. It's a bit difficult to conduct DevOps on data analytical system. Moreover, there are no DevOps solutions for reference. Therefore, exploring DevOps for data analytical system is valuable. In this paper, we illustrate DevOps demands of data analytical system from different perspectives, and constantly emphasize the importance of automation toolchain. Based on them, a process model for DAS DevOps (D2Ops) is proposed to clarify participants activities. In order to improve the efficiency, we attempt to integrate the automation toolchain. With the consid- eration of stability, six generic process components are designed to support this model. They can be the selection criteria for specific automation tools. We also present a reference facility based on these generic process components, and illustrate its implementation combing with a practical case.