AI vs. Human-Authored Headlines: Evaluating the Effectiveness, Trust, and Linguistic Features of ChatGPT-Generated Clickbait and Informative Headlines in Digital News

Abstract

This study explores possible applications of AI technology in online journalism, given the predictions that speed and adaptation to the new medium will increase the penetration of automation in the production business. The literature shows that while the human supervision of journalistic workflow is still considered vital, the journalistic workflow is changing in nature, with the writing of micro-content being entrusted to ChatGPT-3.5 among the most visible features. This research assesses readers’ reactions to different headline styles as tested on a sample of 624 students from Timisoara, Romania, asked to evaluate the qualities of a mix of human-written vs. AI-generated headlines. The results show that AI-generated, informative headlines were perceived by more than half of the respondents as the most trustworthy and representative of the media content. Clickbait headlines, regardless of their source, were considered misleading and rated as manipulative (44.7%). In addition, 54.5% of respondents reported a decrease in trust regarding publications that frequently use clickbait techniques. A linguistic analysis was conducted to grasp the qualities of the headlines that triggered the registered responses. This study provides insights into the potential of AI-enabled tools to reshape headline writing practices in digital journalism.