Calling out policy bullshit: an index for assessing evidence claims in education

Abstract

Policy proposals routinely claim the authority of evidence. This paper argues that the tools available to challenge such claims, although analytically rigorous, often lack the rhetorical capacity to disrupt poorly produced evidence narratives. Using Frankfurt’s account of bullshit and criteria used for evaluating argument, I develop a five-level Policy Bullshit Index (PBI) applicable to any policy document claiming an evidence base–government proposals, commission reports, think-tank briefs, or advocacy documents. The PBI’s distinctive contribution is that it combines defensible scholarly criteria with a rhetoric capable of disrupting evidence-based authority claims that routinely close down scrutiny. I apply the PBI to a Grattan Institute (Australia) report on multi-school organisations which has already been taken up in one state. The analysis shows that the report falls at Level Three (High) on the Index. I propose that the PBI is a potentially useful addition to existing approaches to policy scholarship and policy activism.