Mind the Gaps: How AI Shortcomings and Human Concerns May Disrupt Team Cognition in Human-AI Teams (HATs)

Abstract

As organizations increasingly adopt AI for collaborative work, humans must learn to integrate AI teammates into their workflows. For these human-AI teams (HATs) to function effectively, strong team cognition, encompassing shared understanding, situational awareness, and coordination, is essential. However, unfavorable past experiences, mismatched expectations, and differences in capabilities can make collaboration with AI teammates difficult. To better understand these challenges, we interviewed 30 professionals in high AI-exposure fields to understand their concerns about AI teammates related to team cognition. Participants reported difficulty trusting and relying on AI teammates and expressed fears that overreliance could erode human skills and critical thinking. They also highlighted AI’s limited communication abilities and lack of emotional processing as barriers to effective teamwork. These concerns point to specific behaviors AI teammates must avoid and offer design implications for creating AI teammates that support the effective formation and sustainment of team cognition in HATs.