Bayesian diagnostics for test design and analysis

Abstract

This paper attempts to bridge the gap between classical test theory and item response theory. It is demonstrated that the familiar and popular statistics used in classical test theory can be translated into a Bayesian framework where all of the advantages of the Bayesian paradigm can be realized. In particular, prior opinion can be introduced and inferences can be obtained using posterior distributions. In classical test theory, inferential decisions are based on the values of statistics that are calculated from the responses of subjects over various test questions. In the proposed approach, analogous “statistics” are constructed from the output of simulation from the posterior distribution. This leads to population- based inferences which focus on the properties of the test rather than the performance of specific subjects. The use of the JAGS programming language facilitates extensions to more complex scenarios involving the assessment of tests and questionnaires.