THE PREDICTION OF TOEFL READING COMPREHENSION ITEM DIFFICULTY FOR EXPOSITORY PROSE PASSAGES FOR THREE ITEM TYPES: MAIN IDEA, INFERENCE, AND SUPPORTING IDEA ITEMS

Abstract

The purpose of the current study is to predict the difficulty (equated delta) of a large sample (n=21.3) of TOEFL reading comprehension items. (Only main idea, inference, and supporting statement items were sampled.) A related purpose was to examine whether text and text‐related variables play a significant role in predicting item difficulty; we argued that evidence favoring construct validity would require significant contributions from these particular predictor variables. In addition, details of item predictability were explored by evaluating two hypotheses: (1) that multiple‐choice reading comprehension tests are sensitive to many sentential and discourse variables found to influence comprehension processes in the experimental literature, and (2) that many of the variables identified in the first hypothesis contribute significant independent variance in predicting item difficulty. The great majority of sentential and discourse variables identified in our review of the experimental literature were found to be significantly related to item difficulty within TOEFL's multiple‐choice format. Furthermore, contrary to predictions which we attributed to critics of multiple‐choice tests, the pattern of correlational results showed that there is a significant relationship between item difficulty and the text and text‐related variables. We took this as evidence supporting our claim that multiple–choice reading items yield construct valid measures of comprehension. That is, since critics have pointed out that reading items can often be correctly answered without reading of the text passage, this seems to imply that item variables (not text nor text‐related variables) should fbe prominent predictors of reading item difficulty. Since the contrary relationship was found, we concluded that this provides evidence favoring construct validity. We found, further, in several stepwise linear regression analyses, that many of these text and taxt‐related variables provide independent contributions in predicting reading item difficulty. This was interpreted as providing additional support for construct validity. More specifically, apart from the correlational results, the following stepwise linear regressions results were obtained. For the full sample of 213 items, and where equated delta (an index of item difficulty) is the dependent variable, we found 33 percent (p