A general class of no-regret learning algorithms and game-theoretic equilibria

Abstract

A general class of no-regret learning algorithms, called φ-no-regret learning algorithms is denned, which spans the spectrum from no-internal-regret learning to no-external-regret learning, and beyond. φ describes the set of strategies to which the play of a learning algorithm is compared: a learning algorithm satisfies φ-no-regret iff no regret is experienced for playing as the algorithm prescribes, rather than playing according to any of the transformations of the algorithm's play prescribed by elements of φ. Analogously, a class of game-theoretic equilibria, called φ-equilibria, is denned, and it is shown that the empirical distribution of play of φ-no-regret algorithms converges to the set of φ-equilibria. Perhaps surprisingly, the strongest form of no-regret algorithms in this class are no-internal-regret algorithms. Thus, the tightest game-theoretic solution concept to which φ-no-regret algorithms (provably) converge is correlated equilibrium. In particular, Nash equilibrium is not a necessary outcome of learning via any φ-no-regret learning algorithms.