CoFiSA: A Survey Agent Based on Contrastive Filtering for Ensuring High-Quality Responses in Online Surveys

Abstract

Online survey is a widely used research method in Human-Computer Interaction for its cost-effectiveness and scalability. However, ensuring high-quality responses remains a persistent challenge. While recent work has explored Large Language Model (LLM)-based chatbots to improve engagement, concerns about unreliable data remain. We introduce CoFiSA (Contrastive Filtering Survey Agent), an LLM-based survey agent that generates adaptive follow-up questions in real time based on respondents' inputs. By tailoring questions, CoFiSA encourages responses that meet criteria. We evaluated CoFiSA in a between-subjects study with 48 participants across three conditions: traditional survey, simple feedback loop, and CoFiSA. Responses were scored on five quality dimensions by an LLM evaluator with human verification. CoFiSA outperformed both baselines, producing higher scores in credibility and usefulness, reduced variance, and fewer low-quality responses. This provides empirical evidence that adaptive, contrastive feedback enhances open-ended survey data quality and establishes CoFiSA as a methodological innovation for HCI.