Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting

Abstract

We study a discrete-time non-stationary decision model in which the preferences of the decision maker change over time and are described by quasi-hyperbolic discounting. A time-consistent optimal solution in this model corresponds with a Markov perfect equilibrium in a stochastic game with uncountable state space played by countably many short-lived players. We show that Markov perfect equilibria may be constructed using a generalized policy iteration algorithm. This method is in part inspired by the fundamental works of Mertens and Parthasarathy (in: Raghavan, Ferguson, Parthasarathy, Vrieze (eds) Stochastic games and related topics, Kluwer Academic Publishers, Dordrecht, 1991; in: Neyman, Sorin (eds) Stochastic games and applications, Academic Publishers, Dordrecht, 2003) devoted to subgame perfect equilibria in standard n-person discounted stochastic games. If the one-period utilities and transition probabilities are independent of time, we obtain on new existence results on stationary Markov perfect equilibria in the models with unbounded from above utilities.