USING THE GENERAL DIAGNOSTIC MODEL TO MEASURE LEARNING AND CHANGE IN A LONGITUDINAL LARGE-SCALE ASSESSMENT

Abstract

A general diagnostic model was used to specify and compare two multidimensional item-response-theory (MIRT) models for longitudinal data: (a) a model that handles repeated measurements as multiple, correlated variables over time (Andersen, 1985) and (b) a model that assumes one common variable over time and additional orthogonal variables that quantify the change (Embretson, 1991). Using MIRT-model ability distributions that we allowed to vary across subpopulations defined by type of school, we also compared (a) a model with a single two-dimensional ability distribution to (b) extensions of the Andersen and Embretson approaches, assuming multiple populations. In addition, we specified a hierarchical-mixture distribution variant of the (Andersen and Embretson) MIRT models and compared it to all four of the above alternatives. These four types of models are growth-mixture models that allow for variation of the mixing proportions across clusters in a hierarchically organized sample. To illustrate the models presented in this paper, we applied the models to the PISA-I-Plus data for assessing learning and change across multiple subpopulations. The results indicate that (a) the Embretson-type model with multiple-group assumptions fits the data better than the other models investigated, and (b) the higher performing group shows larger improvement at Time Point 2 than the lower performing group.