Multi-armed Bandit Problem

Abstract

Multi-armed bandit (MAB) problems are a class of sequential resource allo- cation problems concerned with allocating one or more resources among several alternative (competing) projects. Such problems are paradigms of a fun- damental conflict between making decisions (allocating resources) that yield high current rewards, versus making decisions that sacrifice current gains with the prospect of better future rewards. The MAB formulation models resource allocation problems arising in several technological and scientific disciplines such as sensor management, manufacturing systems, economics, queueing and communication networks, clinical trials, control theory, search theory, etc. (see 88 and references therein).