Evaluation of measurement precision with Rasch-type models: The case of the short Generalized Opinion Leadership Scale

Abstract

Item response theory (IRT) forms a flexible psychometric modeling approach to construct and evaluate new assessment instruments. This paper demonstrates how to evaluate the measurement precision of personality scales by assessing their test-retest error and their convergence across different raters within an IRT framework. The first study (N=1575) reports on the construction of a new personality scale to assess generalized opinion leadership by fitting a polytomous Rasch model to the data. Furthermore, in two independent samples the psychometric properties of the scale are evaluated by applying the linear partial credit model. In a retest design the amount of transient error, that is, systematic measurement error specific to a single measurement occasion, is quantified within an IRT framework (study II, N=586) as well as the convergence of self-and observer-ratings of the trait (study III, N=400). © 2010 Elsevier Ltd.