Estimating the reliability of neural network classifications

Abstract

A method for quantifying the reliability of object classifications using trained neural networks is given. Using this method one is able to give an estimation of a confidence value for a certain object. This reveals how trustworthy the classification of the particular object by the neural pattern classifier is. Even for badly trained networks it is possible to give reliable confidence estimations. Several estimators are considered. A k-NN technique has been developed to compare these using a learning set based artificially generated validation set. Experiments show that applying the developed estimators on a validation set gives the same results as applying the estimators on an independent test set. The method was tested on a real-life application, human chromosome classification, and gave good results which indicate the applicability of our method.