Validation of a Multiple Choice English Vocabulary Test with the Rasch Model

Abstract

Validity is the most salient characteristic of any effective measurement instrument. This paper briefly reviews the Messickian validity framework along with its six facets. Then the Rasch model as a measurement model which yields interval scaling in the social sciences is briefly introduced. The contribution that Rasch model can make to establishing different aspects of validity from Messick's view point is discussed and as an illustrative example, the validity of a multiple-choice test of vocabulary is demonstrated using Rasch analysis. The results show that several items misfit the Rasch model. Distractor analysis showed that a few items have distracters which do not function in the intended way and need to be removed. Item-person map showed that the test was on-target and covered a wide range of the ability scale. The performance of the sample on two subsets of the test revealed that participants had identical measures on the two subsets, which is evidence of the unidimensionality of the instrument.