Evaluating Cognitive Biases in Conversational and Generative IIR: A Tutorial

Abstract

Understanding how and why people interact with information through search interfaces is central to Interactive Information Retrieval (IIR). Research has demonstrated that cognitive biases significantly influence search behavior and outcomes. With the latest advances in Generative AI (GenAI), search interfaces are evolving to be more intelligent, adaptive and conversational, offer many new affordances and opportunities. However, they also have the potential to be more constrained, directed and biased. This shift raises important questions about how these advanced, generative and conversational interfaces will affect user behaviours in light of their cognitive biases. This tutorial aims to engage participants in designing experiments to investigate the impact of cognitive biases in the emerging area of Generative IIR (GenIIR). The tutorial is structured into two main sessions. The first half-day will provide an overview of existing research on cognitive biases in information retrieval. We will then explore how advancements in GenIIR create both challenges and opportunities for studying these biases. This foundation will lead into the hands-on segment of the tutorial. Where, in the second half-day, participants will work in groups to design user studies that examine the impact of cognitive biases of users when interacting with GenIIR interfaces.