A framework for mutation testing of machine learning systems

Abstract

In this paper, we provide an insight journey of Testing of Machine Learning Systems (MLS), its evolution, current paradigm, and we propose a machine learning mutation testing framework with scope for future work. Machine Learning (ML) Models are being used even in critical applications such as Healthcare, Automobile, Air Traffic control, Share Trading, etc., and failure of an ML Model can lead to severe consequences in terms of loss of life or property. To remediate this, the ML community around the world, must build highly reliable test architectures for critical ML applications. At the very foundation layer, any test model must satisfy the core testing attributes such as test properties and its components. These attributes should come from the software engineering discipline but the same cannot be applied in as-is form to the ML testing and in this paper, we explain why it is challenging to use Software Engineering Principles as-is when testing any MLS.