Group Differences in the Value of Subscores: A Fairness Issue

Abstract

The aim of this paper was to study fairness in testing by analyzing the quality of subscores for different groups of test takers. This is done by studying the value added ratio (VAR) for all subscores in the test, which here is a Swedish college admission test. Comparisons were made between test takers who take the regular test and test takers who are taking the test with extended time adaptation, as well as between males and females. Significant group differences in such subscore value would raise questions about fairness for that testing program. In particular, differences in subscore value between subgroups based on accommodations could indicate problems with the appropriateness of the accommodation. We found that the value added ratio (VAR) for all subscores in the studied test were slightly larger for males than for females. There were some differences between subgroups with respect to the extended time accommodation, but they were not systematic in the same way as for males and females. While the differences in the subscores' added value were small in general, they may in some instances be large enough to pose a threat to a valid and fair interpretation and use of test scores, and thus raises questions about fairness.