Defining and Measuring News Media Quality: Comparing the Content Perspective and the Audience Perspective

Abstract

High-quality news is important, not only for its own sake but also for its political implications. However, defining, operationalizing, and measuring news media quality is difficult, because evaluative criteria depend upon beliefs about the ideal society, which are inherently contested. This conceptual and methodological paper outlines important considerations for defining news media quality before developing and applying a multimethod approach to measure it. We refer to Giddens' notion of double hermeneutics, which reveals that the ways social scientists understand constructs inevitably interact with the meanings of these constructs shared by people in society. Reflecting the two-way relationship between society and social sciences enables us to recognize news media quality as a dynamic, contingent, and contested construct and, at the same time, to reason our understanding of news media quality, which we derive from Habermas' ideal of deliberative democracy. Moreover, we investigate the Swiss media system to showcase our measurement approach in a repeated data collection from 2017 to 2020. We assess the content quality of fifty news media outlets using four criteria derived from the deliberative ideal (N = 20,931 and 18,559 news articles and broadcasting items, respectively) and compare the results with those from two representative online surveys (N = 2,169 and 2,159 respondents). The high correlations between both methods show that a deliberative understanding of news media quality is anchored in Swiss society and shared by audiences. This paper shall serve as a showcase to reflect and measure news media quality across other countries and media systems.