Gender, knowledge, and trust in artificial intelligence: a classroom-based randomized experiment

Abstract

Artificial intelligence (AI) is increasingly utilized to provide real-time assistance and recommendations across a wide range of tasks in both education and workplace settings, especially since the emergence of Generative AI. However, it is unclear how users perceive the trustworthiness of these tools, particularly given the publicized “hallucinations” that they may experience. We conduct a randomized field experiment in an undergraduate course setting where students perform periodic tests using a digital platform. We analyze how subject characteristics affect trust in AI versus peers’ advice. Students are randomly assigned to either a treatment group receiving advice labeled as coming from an AI system or a control group receiving advice labeled as coming from human peers. Our results are in line with recent laboratory experiments documenting algorithm appreciation. However, this effect is moderated by subject characteristics: male and high-knowledge participants place significantly less weight on AI advice compared to peer advice. Notably, these trust patterns persist regardless of advice quality (correct or incorrect). Moreover, our results remain consistent over a four-week period, including after providing performance feedback during the second week, allowing subjects to make more informed trust decisions.