Markov Decision Processes with a New Optimality Criterion: Discrete Time

Abstract

Standard finite state and action discrete time Markov decision pro- cesses with discounting are studied using a new optimality criterion called moment optimality. A policy is moment optimal if it lexicographically maximizes the sequence of signed moments of total discounted return with a positive (negative) sign if the moment is odd (even). This criterion is equivalent to being a little risk adverse. It is shown that a stationary policy is moment optimal by examining the negative of the Laplace trans- form of the total return random variable. An algorithm to construct all stationary moment optimal policies is developed. The algorithm is shown to be finite.