A potential reduction algorithm for ergodic two-person zero-sum limiting average payoff stochastic games

Abstract

We suggest a new algorithm for two-person zero-sum undis-counted stochastic games focusing on stationary strategies. Given a pos­itive real e, let us call a stochastic game e-ergodic, if its values from any two initial positions differ by at most e. The proposed new algorithm out­puts for every e > 0 in finite time either a pair of stationary strategies for the two players guaranteeing that the values from any initial positions are within an e-range, or identifies two initial positions u and v and cor­responding stationary strategies for the players proving that the game values starting from u and v are at least e/24 apart. In particular, the above result shows that if a stochastic game is e-ergodic, then there are stationary strategies for the players proving 24e-ergodicity. This result strengthens and provides a constructive version of an existential result by Vrieze (1980) claiming that if a stochastic game is 0-ergodic, then there are e-optimal stationary strategies for every e > 0. The suggested algorithm extends the approach recently introduced for stochastic games with perfect information, and is based on the classical potential transfor­mation technique that changes the range of local values at all positions without changing the normal form of the game.