Paradox-Free Analysis for Comparing the Performance of Optimization Algorithms

Abstract

Numerical comparison serves as a major tool in evaluating the performance of optimization algorithms, especially nondeterministic algorithms, but existing methods may suffer from a 'cycle ranking' paradox and/or a 'survival of the nonfittest' paradox. This article searches for paradox-free data analysis methods for numerical comparison. It is discovered that a class of sufficient conditions exist for designing paradox-free analysis. Rigorous modeling and deduction are applied to a class of profile methods employing a filter. It is thus further discovered and proven that algorithm-independent filter conditions can prevent cycle ranking and survival of nonfittest paradoxes from occurring. By adopting an algorithm-independent filter, popular profile methods such as the 'modified data profile method,' 'the accuracy profile method,' and 'the operational characteristics zones method' can be paradox free in comparing or benchmarking the performance of optimization algorithms.