Critical Questions About Scientific Research Publications in the Online Mask Debate

Abstract

Successful management of sociotechnical issues like those raised by the COVID-19 pandemic requires members of the public to use scientific research in their reasoning. In this study, we explore the nature and extent of the public’s abilities to assess research publications through analyzing a corpus of close to 5 K tweets from the early months of the pandemic which mentioned one of six key studies on the then-uncertain topic of the efficacy of face masks. We find that arguers relied on a variety of critical questions to test the adequacy of the research publications to serve as premises in reasoning, their relevance to the issues at hand, and their sufficiency in justifying conclusions. In particular, arguers showed more skill in assessing the authoritativeness of the sources of the publications than in assessing the epistemic qualities of the studies being reported. These results indicate specific areas for interventions to improve reasoning about research publications. Moreover, this study suggests the potential of studying argumentation at the system level in order to document collective preparedness to address sociotechnical issues, i.e., community science literacy.