FallacyCheck - A Proactive LLM-based Browser Extension to Motivate Critical Assessment of News Articles by Questioning Logical Fallacies

Abstract

We present FallacyCheck, a proactive Large Language Model (LLM)-based browser extension designed to motivate the critical assessment of news articles by questioning logical fallacies. Existing LLM-based extensions are reactive, limiting their ability to inoculate users against information disorder. FallacyCheck overcomes this by proactively identifying logical fallacies, such as "Appeal to Emotion"and "Ad Hominem", and most importantly, motivating users to critically assess the article by posing them a thought-provoking, non-leading question in a contextual tooltip. This design, rooted in inoculation theory, prompts users toward meta-cognitive reflection on the argument's logical structure. A preliminary evaluation with 16 highschool students showed the tool was perceived to be easy to use, and the questions helped to stimulate critical thinking. Crucially, participants generally would not have posed these critical questions without the tool's support.