Some comments on representing construct levels in psychometric models

Abstract

This paper is concerned with one of the steps necessary to trace the connection between the substantive theory that serves as a basis for an assessment and the mathematical models that are used to analyze and rate student responses. We are interested in exploring this connection in the context of hypothesized variables that (a) have multiple ordered levels, (b) have been assessed with polytomous items that are meant to capture the aforementioned ordered performance levels, but (c) are modeled as continuous rather than ordinal. We present a straightforward method for estimating interpretable level boundaries when using the partial credit model. We then introduce graphical methods to evaluate the relationship between the levels as estimated by the model and the performance levels originally hypothesized by the theory. We believe that this kind of procedure can help practitioners make meaningful interpretations and provide more accurate diagnostic information to respondents in general.