Variability in Percentage Above Cut Scores Due to Discreteness in Score Scale

Abstract

For standard‐ or criterion‐based assessments, the use of cut scores to indicate mastery, nonmastery, or different levels of skill mastery is very common. As part of performance summary, it is of interest to examine the percentage of examinees at or above the cut scores ( PAC ) and how PAC evolves across administrations. This paper shows that discreteness in score scales can affect the PAC statistics considerably, especially when the test is short and when there are high examinee density and low item density near the cut score on the reporting score scale. This paper also includes recommendations on how to adjust the PAC statistics when they are used in trend analyses. Report Number: ETS RR‐17–32