Measuring Equity-Promoting Behaviors in Digital Teaching Simulations: A Topic Modeling Approach

Abstract

Diversity, equity, and inclusion (DEI) issues are urgent in education. We developed and evaluated a massive open online course (N = 963) with embedded equity simulations that attempted to equip educators with equity teaching practices. Applying a structural topic model (STM)—a type of natural language processing (NLP)—we examined how participants with different equity attitudes responded in simulations. Over a sequence of four simulations, the simulation behavior of participants with less equitable beliefs converged to be more similar with the simulated behavior of participants with more equitable beliefs (ES [effect size] = 1.08 SD). This finding was corroborated by overall changes in equity mindsets (ES = 0.88 SD) and changed in self-reported equity-promoting practices (ES = 0.32 SD). Digital simulations when combined with NLP offer a compelling approach to both teaching about DEI topics and formatively assessing learner behavior in large-scale learning environments.