Validating Score Interpretations Based on Response Processes

Abstract

This chapter focuses on distinguishing protocol and verbal analyses given their origins, empirical support, and differences in collecting, analyzing, and interpreting data. It is essential because it provides readers with a glimpse into what makes these methods different. The chapter focuses on ways in which protocol and verbal analyses inform interpretive and validity arguments. It also focuses on considerations for best practices and areas for future research. Protocol analysis is a method used to collect, analyze, and interpret verbal responses associated with problem-solving processes. It originates from an early psychological method known as experimental self-observation. Protocol and verbal analyses involve collection of data using one-to-one interview sessions. According to Michael Kane, the interpretive argument for indicators of theoretical constructs includes five major inferences - scoring, generalization, extrapolation, theory-based interpretation, and implication. The quality of the evidence, originating from protocol and verbal analyses, comprising the interpretive argument is critical to having confidence in the overall validity argument.