Do We Learn From Each Other: Understanding the Human-AI Co-Learning Process Embedded in Human-AI Collaboration

Abstract

Beyond collaborating in the AI-supported decision-making setting to achieve complementary performance, human and AI should learn from each other and internalize knowledge from their collaboration. This can enhance their individual performance when working independently after their collaboration. However, this expected dual-pathway co-learning process, including both “human learns from AI” and “AI learns from human”, does not occur spontaneously. Human-AI collaboration designs could have inconsistent and intertwined influences on the co-learning process. Based on the learning cycle theory, this study conducted three online, two-stage, and between-subject behavioral experiments to reveal how human and AI learn from each other. By developing a context where human and AI have comparable and moderate performance on emotion classification tasks, our study provides the first empirical evidence of an effective human-AI co-learning process within human-AI collaboration. However, the AI feedback and collaborative workflow design can lead to unequal and potentially negative impacts on both pathways of the co-learning process in groups with varying levels of cognitive reflection capability. These findings highlight three design principles to facilitate the co-learning process embedded in human-AI collaboration rather than naively deploying a complex AI system.