How to Build a Complete Q-Matrix for a Cognitively Diagnostic Test

Abstract

The Q-matrix of a cognitively diagnostic test is said to be complete if it guarantees the identifiability of all possible proficiency classes among examinees. An incomplete Q-matrix causes examinees to be assigned to proficiency classes to which they do not belong. Completeness of the Q-matrix is therefore a key requirement of any cognitively diagnostic test. The importance of the completeness property of the Q-matrix of a test as a fundamental condition to guarantee a reliable estimate of an examinee’s attribute profile has only recently been realized by researchers. In fact, inspection of extant assessments based on the cognitive diagnosis framework often revealed that, in hindsight, the Q-matrices used with these tests were not complete. Thus, the availability of rules for building a complete Q-matrix at the early stages of test development is perhaps at least as desirable as rules for identifying the completeness of a given Q-matrix. This article presents procedures for constructing Q-matrices that are complete. The famous Fraction-Subtraction test problems by K. K. Tatsuoka (1984) are used throughout for illustration.