Dimension-corrected discrimination index

Abstract

The Discrimination Index (DI) or upper-lower index is a simple and robust, classical non-parametric shortcut to evaluate the item discrimination power or item validation in the practical testing settings. Empirical results show that the DI has two shortcomings: first, the traditional DI with the 27% cut-off appears to underestimate for the item discrimination power and, second, for polytomous items, the DI behaves illogically compared to the better performing benchmarking estimators. Therefore, two suggestions are made to modify the procedures when using DI. First, it would be preferable to use the 20% cut-off of instead of 27%; this seems to tend to give estimates that are closer to the better performing association estimators than the point-biserial correlation which is known to underestimate item-score association. Second, two simple modifications of DI, dimension-corrected DI (DI2), are proposed: and, where DI20% refers to the observed value of DI with a 20% cut-off, LN refers to the natural logarithm, and R is the number of categories in the item. Based on the training dataset and external datasets, DI2 seems safe to use at the difficulty levels between p = 0.20–0.80 and when the number of categories in the item does not exceed 7–8.