Sign up & Download
Sign in

Efficacy of effect size measures in logistic regression: An application for detecting DIF

by Juana Gómez-Benito, M. Dolores Hidalgo, José Luis Padilla
Methodology ()

Abstract

Statistical techniques based on logistic regression (LR) are adequate for the detection of differential item functioning (DIF) in dichotomous items. Nevertheless, they return more false positives (FPs) than do other DIF detection techniques. This paper compares the efficacy of DIF detection using the LR significance test and the estimation of the effect size that these procedures provide using R² of Nagelkerke. The variables manipulated were different conditions of sample size, focal and reference group sample size ratio, amount of DIF, test length and percentage of test items with DIF. In addition, examinee responses were generated to simulate both uniform and nonuniform DIF (symmetric and asymmetric). In all cases, dichotomous response tests were used. The results show that the use of R² as a strategy for detecting DIF obtained lower correct detection percentages than those obtained from significance tests. Moreover, the LR significance test showed adequate control of FP rates, close to the nominal 5%, although the rate was slightly higher than the nominal 5% when the sample size was smaller. However, when the effect size measure was used to detect DIF, the FP rates were lower and <1% for a wide number of conditions. In addition, a statistically significant main effect of the sample size variable was obtained. Thus, the FP percentages were higher when the sample size was small (100/100). The results obtained indicate that the use of R² as a measure of effect size together with the statistical significance test reduces the rate of FP.

Cite this document (BETA)

Readership Statistics

16 Readers on Mendeley
by Discipline
 
 
 
by Academic Status
 
31% Associate Professor
 
25% Ph.D. Student
 
19% Researcher (at an Academic Institution)
by Country
 
13% United States
 
6% Brazil

Sign up today - FREE

Mendeley saves you time finding and organizing research. Learn more

  • All your research in one place
  • Add and import papers easily
  • Access it anywhere, anytime

Start using Mendeley in seconds!

Already have an account? Sign in