Evaluating reward definitions for parameter control

Abstract

Parameter controllers for Evolutionary Algorithms (EAs) deal with adjusting parameter values during an evolutionary run. Many ad hoc approaches have been presented for parameter control, but few generic parameter controllers exist. Recently, successful parameter control methods based on Reinforcement Learning (RL) have been suggested for one-off applications, i.e. relatively long runs with controllers used out-of-the-box with no tailoring to the problem at hand. However, the reward function used was not investigated in depth, though it is a non-trivial factor with an important impact on the performance of a RL mechanism. In this paper, we address this issue by defining and comparing four alternative reward functions for such generic and RL-based EA parameter controllers. We conducted experiments with different EAs, test problems and controllers and results showed that the simplest reward function performs at least as well as the others, making it an ideal choice for generic out-of-the-box parameter control.