Differentiating response styles and construct-related responses: A new IRT approach using bifactor and second-order models

Abstract

Response styles (RS) can interfere if rating or Likert scales are used to assess personality traits, biasing survey results and affecting the validity of the measurement. Based on a new approach using multidimensional item response theory (MIRT) models (Böckenholt, U., Psychological Methods, 2012, doi:10.1037/a0028111), rating data can be analyzed and RS can be controlled for by a decomposition of the ordinal response into different sequential processes using a binary decision tree. Expanding on this approach (using multiscale questionnaires), Khorramdel and von Davier (in press) showed that the obtained RS measures are not always unidimensional but may be confounded with trait-related responses. The current paper addresses this problem by applying bifactor and second-order item response theory (IRT) models to pseudo items, examining extreme RS (ERS) and midpoint RS (MRS). In contrast to simple-structure MIRT models, these models allow item responses to depend on two factors: a general factor (RS) and a specific factor (personality trait). Findings from two empirical applications using questionnaires which measure the Big Five personality dimensions and a five-point Likert scale show a superior model fit of the bifactor model with regard to ERS and MRS. While the ERS measure clearly appears to be a distinct RS measure, the MRS appears not to be, but both response style measurements show high model-based reliabilities.