On Optimal Ordering in the Optimal Stopping Problem

Abstract

Consider a player who can probe a sequence of n independent random variables X1, . . . , Xn with known distributions. After observing (the realized value of) Xi, the player needs to decide whether to stop and earn reward Xi, or reject the reward and probe the next variable Xi+1. The goal is to maximize the expected reward at the stopping time. This is an instance of the optimal stopping problem, which is a fundamental problem studied from many different aspects in mathematics, statistics, and computer science, and has found a wide variety of applications in sequential decision making and mechanism design. When the order in which the random variables X1,...,Xnare probed is fixed, the optimal stopping strategy can be found by solving a simple dynamic program. Under this strategy, at every step i, the player would compare the realized value of the current random variable Xi to the expected reward (under the optimal strategy for the remaining subproblem) from the remaining variables Xi+1, . . .,Xn, and stop if the former is greater than the latter. In this paper, we focus on the relatively less studied question of optimizing the order in which the random variables should be probed. Specifically, besides choosing a stopping strategy, if the player is free to choose the order in which the random variables are probed, then which ordering would maximize the expected reward at the stopping time? The optimal ordering problem has been previously studied in mathematics and statistics literature in the 80's (e.g., Gilat[6], Hill and Hordijk[8], and Hill[7]). However the focus there has been on analytically characterizing the optimal order for some special structured cases (like Bernoulli and exponential distributions). One difficulty in such a study is that the nature of this problem changes significantly depending on the type of distributions considered. For example, when distributions are Bernoulli or exponential, the optimal ordering can be found analytically [8], but, the problem remains nontrivial for uniform distributions.