Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

Abstract

In this paper, we consider a two-level multidimensional item response model that examines country differences in extreme response style (ERS) as a possible cause for the achievement-attitude paradox in PISA 2006. The model is an extension of Bolt & Newton (2011) that uses response data from seven attitudinal scales to assess response style and to control for its effects in estimating correlations between attitudes and achievement. Despite detectable variability in ERS across countries and detectable biasing effects of ERS on attitudinal scores, our results suggest that the unexpected between-country correlation between attitudes and achievement is not attributable to country differences in ERS. The remaining between-country correlations between mean attitudes and mean achievement once controlling for ERS can be explained by the observation that (1) despite detectable country differences, most variability in ERS occurs within, as opposed to between, countries, and (2) ERS appears to be only weakly correlated with achievement. The methodological approach used in this paper is argued to provide an informative way of studying the effects (or lack thereof) of cross-country variability in response style.