A Vision on accelerating enterprise IT system 2.0

Abstract

The proliferation of commodity based big data platforms and an exponential increase in the research in machine learning techniques lead to a change in application development paradigm from traditional control-flow Software 1.0 to data-flow Software 2.0 programming paradigm e.g. use of machine learning based models over customer-scoring methods for generating recommendations. The Software 2.0 paradigm is a data-driven programming that requires specialized data management to get clean, governed and unbiased data sets, well defined neural network architectures for building a model, efficient model training, extensive testing and high performance deployment. Unlike Software 1.0 paradigm, a Software 2.0 program's output is probabilistic in nature as the correctness is highly dependent on the size and quality of the input data, however the program's performance is deterministic. This has led to the research in specialized hardware and high performance architectures for deep-learning algorithms. Also, the nature of Software 2.0 paradigm brings in heterogeneity in the whole life cycle starting from an application development until its deployment in the production environment and hence posing numerous architecture and performance challenges. In this paper, we outline the research problems that will emerge due to migration of a part of Software 1.0 to Software 2.0.We present the challenges and the approaches to address them, for accelerating the development and deployment of Software 2.0 programs. We also envision evolution of existing enterprise IT systems to the data-driven enterprise IT systems, referred to as EIT 2.0. We have compared a conventional development life-cycle of applications with that in EIT 2.0. We address research problems and approaches with the related state-of-art in the performance engineering of modern enterprise applications during its life cycle in EIT 2.0.