Sources of differential item functioning between Korean and Japanese examinees on a second language vocabulary test

Abstract

The use of item response theory in equating or creating computeradaptive tests relies on the assumption of invariance of item parameters across populations. This assumption can be assessed with an analysis of differential item functioning (DIF). The purpose of this study was (a) to ascertain whether DIF between two native language groups was present on a 90-item multiple-choice English vocabulary test and (b) to explore the causes of DIF, should it exist. Participants were 184 Korean and 146 Japanese undergraduate students learning English as a foreign language in their home countries. A separate calibration t-test approach was used to identify DIF, with the criteria set at p B 0.01 and effect size 1 logit, calculated as the difference in Rasch item-difficulty between the two groups. Twenty-one items displayed DIF. The causes of DIF in nine of those items were tentatively identified as relating to their status as loanwords in the L1. When a tested word was a loanword in both Korean and Japanese, differences in both the frequency and range of use of the loanword in the two languages predicted the direction of DIF. Similarly, phonological/orthographic overlap between two separate English loanwords in the L1 was found to be a possible cause of DIF. Implications for test development and further research in this area are discussed.