MAP-Elites with Cosine-Similarity for Evolutionary Ensemble Learning

Abstract

Evolutionary ensemble learning methods with Genetic Programming have achieved remarkable results on regression and classification tasks by employing quality-diversity optimization techniques like MAP-Elites and Neuro-MAP-Elites. The MAP-Elites algorithm uses dimensionality reduction methods, such as variational auto-encoders, to reduce the high-dimensional semantic space of genetic programming to a two-dimensional behavioral space. Then, it constructs a grid of high-quality and diverse models to form an ensemble model. In MAP-Elites, however, variational auto-encoders rely on Euclidean space topology, which is not effective at preserving high-quality individuals. To solve this problem, this paper proposes a principal component analysis method based on a cosine-kernel for dimensionality reduction. In order to deal with unbalanced distributions of good individuals, we propose a zero-cost reference points synthesizing method. Experimental results on 108 datasets show that combining principal component analysis using a cosine kernel with reference points significantly improves the performance of the MAP-Elites evolutionary ensemble learning algorithm.