The precision of students' ability estimation on combinations of item response theory models

Abstract

This study compared the precision of ability estimation on different types of item response theory models for mixed-format data. Participants in this study were 1625 Junior High School Students in Depok, Indonesia. The mixed-format test was used to measure the students' ability in mathematics. The test used consists of multiplechoice and constructed response. Multiple-choice items are scored dichotomously, whereas constructed response items are scored polytomously. Furthermore, the mixed response data were analyzed using combinations of item response theory models. This study used a combination of Multiple-Choice Model for dichotomous data and Graded response model for polytomous data (MCM+GRM). Analysis of this model combination has never been done simultaneously. Test response data were analysed using PARSCALE. Furthermore, the estimation results were compared with the estimation results from a combination of 3 Parameters Logistic Model and Generalized Partial Credit Model (3PLM+GPCM). There are two criteria evaluation for the level of estimation precision: Root Mean Squared Error (RMSE) and correlation method. Based on the results obtained, the estimated RMSE value for the MCM+GRM is smaller than the estimated RMSE value with the 3PLM+GPCM. Also, the results of the estimated ability with MCM+GRM produce higher correlation values than 3PLM+GPCM. So, it can be concluded that the level precision of the MCM+GRM model is higher than 3PLM+GPCM. Therefore, MCM+GRM is more recommended for estimating students' mathematical ability in mixed-format tests.