The application of multidimensional rasch models in large scale assessment and validation: An empirical example

Abstract

Introduction: Accountability and diagnostic assessments usually comprise several subtests to provide information on the different components of the abilities tested. For practicality reasons such subtests are usually short. The major drawback of short subtests is their low reliability and imprecision. This study shows how a compensatory multidimensional Rasch model, namely, Multidimensional Random Coefficient Multinomial Logit Model (MRCMLM), can utilize the correlations among subtests to improve estimation and the reliability of measures. Method: The sample was composed of 1021 Iranian high school students. An English language comprehension test composed of four 10-item subtests was adminstretd to the participants. The data were analysed with both MRCMLM and unidimensional Rasch model. Results: Findings showed that subtests that contain only ten items can have the reliability of 45-item tests when students' abilities are estimated with the MRCMLM. It is also demonstrated that multidimensional analysis gives a better estimate of the true correlations among subtests and can be used as a confirmatory approach to study the componential structure of proficiency constructs. Conclusions: Multidimensional Rasch model can improve the reliability of short subtests. The model can also be used as a confirmatory approach to study the componential structure of proficiency constructs and to show the true association among dimensions of profificency. © Education & Psychology I+D+i and Editorial EOS (Spain).