AN INQUIRY INTO THE NATURE OF THE SENTENCE-COMPLETION TASK: IMPLICATIONS FOR ITEM GENERATION

Abstract

The purpose of this two-part study was to examine the cognitive skills underlying performance on Graduate Record Examinations (GRE®) sentence-completion items. In the first investigation, a database of 112 operationally administered items was used to generate validity information for a subset of hypothesized skills that had been shown to be useful in previous research. Surprisingly, very few of the hypothesized skills performed as expected. In the second investigation, a subset of 45 item variants was used to (a) verify the results of the first investigation; (b) generate new hypotheses about the combinations of item-stimulus features that signal particular required skills; and (c) provide an initial test of those hypotheses. This second investigation verified the findings of the first investigation and provided additional insights into the nature of the sentence completion task In particular, the analysis yielded detailed information about (a) the mental representations constructed by students during the process of item solving, (b) procedures for determining the subsets of item words that are actually considered by students during the process of item solving, and (c) approaches for predicting the operating characteristics of items when multiple solution strategies are possible. These findings will facilitate subsequent efforts to write new test items that target specific difficulty levels and to create performance summaries that are indicative of students' strengths and weaknesses.