Online learning with an almost perfect expert

Abstract

We study multiclass online learning, where a forecaster predicts a sequence of elements drawn from a finite set using the advice of n experts. Our main contributions are to analyze the scenario where the best expert makes a bounded number b of mistakes and to show that, in the low-error regime where b = o(log n), the expected number of mistakes made by the optimal forecaster is at most log 4 n + o(log n). We also describe an adversary strategy showing that this bound is tight and that the worst case is attained for binary prediction.