Towards an improved strategy for solving Multi-Armed Bandit problem

Abstract

Multi-Armed Bandit (MAB) problem is one of the classical reinforcements learning problems that describe the friction between the agent’s exploration and exploitation. This study explores metaheuristics as optimization strategies to support Epsilon greedy in achieving an improved reward maximization strategy in MAB. In view of this, Annealing Epsilon greedy is adapted and PSO Epsilon greedy strategy is newly introduced. These two metaheuristics-based MAB strategies are implemented with input parameters, such as number of slot machines, number of iterations, and epsilon values, to investigate the maximized rewards under different conditions. This study found that rewards maximized increase as the number of iterations increase, except in PSO Epsilon Greedy where there is a non-linear behavior. Our Annealing-Epsilon greedy strategy performed better than Epsilon Greedy when the number of slot machines is 10, but Epsilon greedy did better when the number of slot machines is 5. At the optimal value of Epsilon, which we found at 0.06, Annealing Epsilon greedy performed better than Epsilon greedy when the number of iterations is 1000. But at number of iterations ≥ 1000, Epsilon greedy performed better than Annealing Epsilon greedy. A stable reward maximization values are observed for Epsilon greedy strategy within Epsilon values 0.02 and 0.1, and a drastic decline at epsilon > 0.1.