Normality from monte carlo simulation for statistical validation of computer intensive algorithms

Abstract

The latest AI techniques are usually computer intensive, as opposed to the traditional ones which rely on the consistency of the logic principles on which they are based. In contrast, many algorithms of Computational Intelligence (CI) are meta-heuristic, i.e. methods where the particular selection of parameters defines the details and characteristics of the heuristic proper. In this paper we discuss a method which allows us to ascertain, with high statistical significance, the relative performance of several meta-heuristics. To achieve our goal we must find a statistical goodness-of-fit (gof) test which allows us to determine the moment when the sample becomes normal. Most statistical gof tests are designed to reject the null hypothesis (i.e. the samples do NOT fit the same distribution). In this case we wish to determine the case where the sample IS normal. Using a Monte Carlo simulation we are able to find a practical gof test to this effect. We discuss the methodology and describe its application to the analysis of three case studies: training of neural networks, genetic algorithms and unsupervised clustering.