Mathematical Ranking of AI Classifiers Using Confusion Matrix and Matthews Correlation Coefficient

Abstract

This research primarily deals with mathematically ranking the level of accuracy of various Artificial Intelligence (AI) based machine learning classifiers, using Mathews Correlation Coefficient (MCC), leveraging Confusion Matrices. A detailed Literature survey was done to gather the existing knowledge. This knowledge was used as foundational basis to further build the scope of this research project. The classifiers used were Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Convolutional Neural Network (CNN). These classifiers were used for Face recognition of individuals. A total of 33 adult test subjects with 10 images per subject resulting in a total of 330 distinct images were part of this research project. These test subjects were from the Labeled Faces in The Wild publicly available database along with teachers from James B. Conant High School who voluntarily participated in this research. Python based face recognition program wrappers and associated environment was built around pre-existing classifiers and the image data was passed to these wrappers. These wrappers recognized the faces of individuals in the images over 10 trials wherein each trial consisted of 33 distinct images, with varying degree of accuracy and presented that as outputs. These outputs were used to determine the Confusion Matrices which in turn were used to calculate the MCC scores. The MCC scores were plotted, and these results showed that the SVM AI classifier had the highest level of relative accuracy for face recognition, followed by KNN and CNN classifiers.