A Bandit You Can Trust

Abstract

This work proposes Dynamic Linear Epsilon-Greedy, a novel contextual multi-armed bandit algorithm that can adaptively assign personalized content to users while enabling unbiased statistical analysis.Traditional A/B testing and reinforcement learning approaches have trade-offs between empirical investigation and maximal impact on users.Our algorithm seeks to balance these objectives, allowing platforms to personalize content effectively while still gathering valuable data.Dynamic Linear Epsilon-Greedy was evaluated via simulation and an empirical study in the ASSISTments online learning platform.In simulation, Dynamic Linear Epsilon-Greedy performed comparably to existing algorithms and in ASSISTments, slightly increased students’ learning compared to A/B testing.Data collected from its recommendations allowed for the identification of qualitative interactions, which showed high and low knowledge students benefited from different content.Dynamic Linear Epsilon-Greedy holds promise as a method to balance personalization with unbiased statistical analysis.All the data collected during the simulation and empirical study are publicly available at https://osf.io/zuwf7/.