Relationship between types of distractor and difficulty of multiple-choice vocabulary tests in sentential context

Abstract

This study compares the effect of different kinds of distractors on the level of difficulty of multiple-choice (MC) vocabulary tests in sentential contexts. This type of test is widely used in practical testing but it has received little attention so far. Furthermore, although distractors, which represent the unique characteristics of MC tests, are known to influence test difficulty, studies have focused only on the semantic relationship between target words and distractors in vocabulary tests. By also considering the words that are syntagmatically related to the words in context, this study contrasted distractors relating to target words and those relating to context characteristics, and compared three MC vocabulary tests with different types of distractors: distractors with a paradigmatic relationship to the words in the correct answer, distractors with a syntagmatic relationship to contexts, and unrelated distractors. The results suggested that tests with syntagmatically related distractors were generally the most difficult, and tests with unrelated distractors, the easiest; the paradigmatically related distractors remained in the middle. However, this difference disappeared when the test takers could not use contextual information, which indicates that test takers strongly rely on contextual information in taking multiple-choice vocabulary tests in context.