Online Optimization—An Introduction

Abstract

It has been widely recognized (and in some cases for quite a long time) within various communities that there are many situations in which present actions must be made and resources allocated with incomplete knowledge of the future. The difficulty in these situations is that we may have to make our decision based only on the past and the current task we have to perform. It is not even clear how to measure the quality of a proposed decision strategy. The approach usually taken is to devise some probabilis- tic model of the future and act on this basis. This was the starting point of the theory of Markov decision processes. Other recent approaches—all under the same name, robust optimization—have been proposed, where uncertainty is characterized by set membership, rather than distributions. Online optimization is a different approach, popularized within computer science, and aims at comparing the performance of a strategy that operates with no knowledge of the future (online) with the performance of an optimal strategy that has complete knowledge of the future (offline). In this tuto- rial, we are proposing to give an overview of the state of the art of this approach—its applications, tools, and techniques—highlighting the relevance to operations research and management science.