A Measurement Gap? Effect of Survey Instrument and Scoring on the Partisan Knowledge Gap

Abstract

Research suggests that partisan gaps in political knowledge with partisan implications are wide and widespread in the United States. Using a series of experiments, we estimate the extent to which the partisan gaps in commercial surveys reflect differences in confidently held beliefs rather than motivated guessing. Knowledge items on commercial surveys often have guessing-encouraging features. Removing such features yields scales with greater reliability and higher criterion validity. More substantively, partisan gaps on scales without these “inflationary” features are roughly 40 percent smaller. Thus, contrary to some prior research, which finds that the upward bias is explained by the knowledgeable deliberately marking the wrong answer (partisan cheerleading), our data suggest that partisan gaps on commercial surveys in the United States are strongly upwardly biased by motivated guessing by the ignorant. Relatedly, we also find that partisans know less than what toplines of commercial polls suggest.