Designing Hybrid Human-AI Orchestration Tools for Individual and Collaborative Activities: A Technology Probe Study

Abstract

Combining individual and collaborative learning is common, but dynamic combinations (which happen as-the-need arises, rather than in preplanned ways, and may happen on an individual basis) are rare. This work reports findings from a technology probe study exploring alternative designs for classroom co-orchestration support for dynamically transitioning between individual and collaborative learning. The study involved 1) a technology-probe classroom study in an authentic, AI-supported classroom to understand teachers' and students' needs for co-orchestration support over dynamic transitions; and 2) workshops and interviews with students and teachers to get informed feedback about their lived experiences. About 118 students and 3 teachers from a middle school in the US experienced a pairing policy - student, teacher, and AI-controlled pairing policy - (i.e., identifying students needing help and potential helpers) for switching from individual to a peer-tutoring activity. This work aims to answer the following questions: 1) How did students and teachers react to these pairing policies? and 2) What are students' and teachers' desires for sharing control over the orchestration of dynamic transitions? Findings suggest the need for a form of hybrid control between students, teachers, and AI systems over transitions, as well as for adaptivity and adaptability for different classroom characteristics, teachers, and students' prior knowledge.