Feeds of Distrust: Investigating How AI-Powered News Chatbots Shape User Trust and Perceptions

Abstract

The start of the 2020s ushered in a new era of AI through the rise of Generative AI Large Language Models (LLMs) such as ChatGPT. These AI chatbots offer a form of interactive agency by enabling users to ask questions and query for more information. However, prior research only considers if LLMs have a political bias or agenda, and not how a biased LLM can impact a user’s opinion and trust. Our study bridges this gap by investigating a scenario where users read online news articles and then engage with an interactive AI chatbot, where both the news and the AI are biased to hold a particular stance on a news topic. Interestingly, participants were far more likely to adopt the narrative of a biased chatbot over news articles with an opposing stance. Participants were also substantially more inclined to adopt the chatbot’s narrative if its stance aligned with the news—all compared to a control news-article only group. Our findings suggest that the very interactive agency offered by an AI chatbot significantly enhances its perceived trust and persuasive ability compared to the ‘static’ articles from established news outlets, raising concerns about the potential for AI-driven indoctrination. We outline the reasons behind this phenomenon and conclude with the implications of biased LLMs for HCI research, as well as the risks of Generative AI undermining democratic integrity through AI-driven Information Warfare.