Validation of machine learning classifiers using metamorphic testing and feature selection techniques

Abstract

Testing involves examining the behavior of a system in order to discover potential faults. Given an input for a system, the challenge of distinguishing the correct behavior from potentially incorrect one, is called the “test oracle problem”. Metamorphic testing has shown great potential in overcoming the test oracle problem. In this work, we apply metamorphic testing to validate experimentally machine learning classification algorithms, namely Naïve Bayes (NB) and k-Nearest Neighbor (k-NN) individually and in combination (i.e., ensemble classifications methods), using real-world biomedical datasets. Furthermore, advanced feature selection techniques and synthetic minority over-sampling technique (SMOTE) are used in order to generate our test suite and meet the requirements of the specified metamorphic relations. While, this study reveal that NB and k-NN satisfy the specified metamorphic relations, it also concludes that it is not compulsory that the metamorphic relations that are necessary for NB and k-NN individually, are also necessary for their ensemble classifier.