Computational Psychometrics for Digital-First Assessments: A Blend of ML and Psychometrics for Item Generation and Scoring

Abstract

With the growth of digital technology and advances in automated test development tools, ranging from automated item generation to automated scoring, there is tremendous opportunity to develop innovative forms of technology-based assessments. In this chapter we will describe and discuss how these machine learning (ML) techniques are used for assessment development and scoring while satisfying the psychometric constraints for reliability, targeted difficulty, and validity. The concept of computational psychometrics is briefly introduced as a framework in which the algorithms and psychometric models are combined to support the test’s validity, reliability, and generalizability. We will illustrate these concepts using the Duolingo English Test.