Comparison and Selection of Five Noncompensatory Cognitive Diagnosis Models Based on Attribute Hierarchy Structure

Abstract

Attribute hierarchy structure (AHS), which was considered as the basis of cognitive diagnosis, could largely affect the classification accuracy. However, in practical work, it was very difficult to determine whether the specified AHS was rational or not. Thus, it is necessary to explore how the AHS will affect the classification accuracy. This paper investigated the effect of different AHSs on the accuracy of diagnosis. And two AHSs were under investigation, one is the correctly identified AHS and the other is the incorrectly identified AHS. The commonly used Monte Carlo simulation method was employed to generate the data. And five cognitive diagnostic models. Rule Space Model (RSM), Attribute Hierarchy Model (AHM), General Distance Decision (GDD) Model, DINA_HC model and DINA model, were used to fit the same data. The results indicated that: (1) When the AHS was correctly identified, the attribute match ratios (AMRs) under RSM and AHM were both relatively low, while the AMRs under GDD, DINA_HC and DINA models were all relatively high. Furthermore, the AMRs under DINA_HC and DINA models were larger than that of GDD model. (2) When the AHS was incorrectly identified, the AMRs under RSM, AHM and GDD models were all relatively smaller compared to the case in which AHS was correctly identified, which indicated that the AHS significantly affects the accuracy of diagnosis of the RSM, AHM and GDD models. On the other hand, the influence of AHS on the accuracy of the diagnosis of DINA_HC model was moderate. But the accuracy of the diagnosis of DINA model will not be influenced by the AHS because the AHS information was not used in the DINA model. (PsycINFO Database Record (c) 2016 APA, all rights reserved)