SourceFinder: A Construct-Driven Approach for Locating Appropriately Targeted Reading Comprehension Source Texts

Abstract

A fully-automated approach for locating source material for use in developing reading comprehension/verbal reasoning passages is described. The system employs a combination of classification and regression techniques to predict the acceptability status of candidate source texts downloaded from targeted on-line journals and magazines. The approach is applied to the problem of selecting source texts pitched at a particularly advanced reading level, i.e., the level expected for students seeking admission to graduate school. Results confirm that, even at this advanced level, SourceFinder behaves much like a human rater. In particular, while the human raters agreed with each other 63% of the time, the agreement between SourceFinder and a human rater ranged from 61% to 62%. This suggests that the estimated models have succeeded in capturing useful information about the characteristics of texts that affect test developers’ ratings of source acceptability and that continued use of the system may help test developers find more high quality sources in less time.