Online administration of a reasoning inventory in development

Abstract

We are developing the Physics Inventory of Quantitative Literacy (PIQL), a new research based assessment (RBA) focused on quantitative reasoning—rather than conceptual understanding—in physics contexts. We rapidly moved administration of the PIQL online in Spring 2020 due to the COVID-19 pandemic. We present our experiences with online, unproctored administration of an RBA in development to students enrolled in a large-enrollment, calculus-based, introductory physics course. We describe our attempts to adhere to best practices on a limited time frame, and present a preliminary analysis of the results, comparing results from the online administration to earlier results from in-person, proctored administration. We include discussion of online administration of multiple-choice/multiple-response (MCMR) items, which we use on the instrument as a way to probe multiple facets of student reasoning. Our initial comparison indicates little difference between online and paper administrations of the PIQL, except for performance MCMR items