A MCDM-Based Evaluation Approach for Imbalanced Classification Methods in Financial Risk Prediction

Abstract

Various classifiers have been proposed for financial risk prediction. The traditional practice of using a singular performance metric for classifier evaluation is not sufficient for imbalanced classification. This paper proposes a multi-criteria decision making (MCDM)-based approach to evaluate imbalanced classifiers in credit and bankruptcy risk prediction by considering multiple performance metrics simultaneously. An experimental study is designed to provide a comprehensive evaluation of imbalanced classifiers using the proposed evaluation approach over seven financial imbalanced data sets from the UCI Machine Learning Repository. The TOPSIS, a well-known MCDM method, was applied to rank three categories of imbalanced classifiers using six popular evaluation criteria. The rankings results indicate that: 1) the rankings generated by the TOPSIS, which combine the results of six evaluation criteria, provide a more reasonable evaluation of imbalanced classifiers over any single performance criterion; and 2) Synthetic Minority Oversampling Technique (SMOTE)-based ensemble techniques outperform other groups of imbalanced learning approaches. Specifically, SMOTEBoost-C4.5, SMOTE-C4.5, and SMOTE-MLP were ranked as the top three classifiers based on their performances on the six criteria.