Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

Abstract

Gender gaps on the most widely used conceptual inventories created by physics education researchers have been extensively studied. Most of the research exploring the consistent gender gaps has been performed at the student level using the total evaluation score; less research has been performed examining these assessments at the item level and this research has been predominately restricted to the Force Concept Inventory (FCI). Many studies have identified subsets of FCI items as unfair to either men or women. An item is fair if men and women of equal ability in conceptual physics score equally on the item. This study explored the item-level gender fairness of the Force and Motion Conceptual Evaluation (FMCE) and the Conceptual Survey of Electricity and Magnetism (CSEM). Classical test theory and differential item functioning (DIF) analysis were employed to examine item fairness. Fairness was investigated with four large post-Test samples, two for the FMCE (n1=3016, n2=3719) and two for the CSEM (n1=2014, n2=2657). Men and women performed significantly differently on the majority of FMCE items but with no more than a small effect size. There were fewer items in the CSEM where men and women performed differently. Using DIF analysis, which assumes that overall test score is an accurate measure of ability, only one item in the FMCE demonstrated large DIF in both samples with that item unfair to women. One additional item showed large DIF in a single sample, also unfair to women. Only one item in the CSEM demonstrated large DIF. The item was unfair to men but this result was not consistent across all samples. The number of large DIF items identified in both the FMCE and the CSEM was substantially smaller than the number of large DIF items identified in the FCI by previous studies.