QuaRCS Light: Validation of an Abbreviated Assessment of Undergraduate Numeracy and Mathematics Affect

Abstract

In this study, we present evidence for the validity of a shortened form of the Quantitative Reasoning for College Science (QuaRCS) Assessment, a validated instrument assessing the numeracy and math-related affect of undergraduate students in general education/introductory science courses. Previously published analyses of QuaRCS data revealed that 1) roughly 30% of students found the assessment boring, leading to lower self-reported effort and 2) affective factors (e.g. numerical self-efficacy) were significant predictors of QuaRCS score. As a result, we reduced the length of the assessment from 25 to 15 quantitative items, and expanded the affective variable selection from three to eight to include math related anxiety, situational math affect, sense of belonging, growth mindset and metacognition. We administered the abbreviated assessment (“QuaRCS light”) to roughly 15,000 students across 18 institutions and validated it with classical test theory and item response theory based methods. We found, despite a modest decrease in reliability, students' effort scores were significantly higher on QuaRCS light, justifying this tradeoff. In addition, we validated the new affective factors using exploratory and confirmatory factor analysis. When included in a linear regression model as predictors of QuaRCS score, these 8 factors explain 31% of the observed score variance, increasing to 49% when student confidence, effort, and calculator usage are included. Our findings emphasize the importance of affective factors in understanding and fostering numeracy, and this work informs the design of more holistic and effective assessments that are appropriate for assessing numeracy in diverse student populations.