Are Online and Paper Tests Comparable? Evidence from Statewide K-12 Tests

Abstract

We investigate two research questions using a recent statewide transition from paper to computer-based testing: first, the extent to which test mode effects found in prior studies can be eliminated; and second, the degree to which online and paper assessments offer different information about underlying student ability. We first find very small mode effects for a more recent transition in Massachusetts. Second, we investigate the predictive evidence of validity for paper and online tests for predictions of future test scores and grades. We generally find minimal differences for the extent to which scores on paper tests can differentially predict future online versus paper test scores. Finally, online and paper test scores are similarly predictive of future grade point average. We conclude that the online test penalty can vary substantially by test and that extreme care should be taken when administering online tests to some students and paper tests to others.