Development and validation of a measure of authentic online work

Abstract

Researchers tout digital learning as a tool that can increase the authenticity of student learning and assessment tasks but lack a psychometrically valid instrument to test this hypothesis. Further, there are several complementary definitions of authentic work, versus a single agreed upon definition, presented in academic literature. I synthesized this literature to develop the Authentic Online Work Rubric that measures two primary components of authentic online work – opportunities for higher-order thinking and real-world relevance. Data were collected from online courses developed by one of the largest online course providers in the United States. I validated the scale using principal component analysis before generating a lesson-level standardized coefficient using item response theory for both the higher-order thinking and real-world relevance subscales. The use of this rubric facilitates the measurement of authentic work and is targeted for use in the evaluation of learning tasks and assessments in online instructional settings to support researcher, developer, and school-based continuous improvement processes.