Investigating Sources of Differential Item Functioning in International Large-Scale Assessments Using a Confirmatory Approach

  • Sandilands D
  • Oliveri M
  • Zumbo B
 et al. 
  • 20

    Readers

    Mendeley users who have this article in their library.
  • 5

    Citations

    Citations of this article.

Abstract

International large-scale assessments of achievement often have a large degree of differential item functioning (DIF) between countries, which can threaten score equivalence and reduce the validity of inferences based on comparisons of group performances. It is important to understand potential sources of DIF to improve the validity of future assessments; however, previous attempts to identify sources of DIF have had variable results. This study had two purposes. The first was to apply a confirmatory approach (Poly-SIBTEST) to investigate sources of DIF typically found in international large-scale assessments: adaptation effects and cognitive loadings of items. We conducted three pairwise DIF analyses on Spanish and English versions of the Progress in International Reading Literacy Study 2001 Reader booklet. Results confirmed that item cognitive loadings were a source of differential functioning favoring both England and the United States when compared against Colombia; however, adaptation effects did not consistently favor one group or the other. The second purpose of this study was to highlight strengths and limitations of Poly-SIBTEST for conducting substantive analyses of differential functioning sources and also to offer suggestions for future directions on this type of methodological research. International large-scale assessments of achievement often have a large degree of differential item functioning (DIF) between countries, which can threaten score equivalence and reduce the validity of inferences based on comparisons of group performances. It is important to understand potential sources of DIF to improve the validity of future assessments; however, previous attempts to identify sources of DIF have had variable results. This study had two purposes. The first was to apply a confirmatory approach (Poly-SIBTEST) to investigate sources of DIF typically found in international large-scale assessments: adaptation effects and cognitive loadings of items. We conducted three pairwise DIF analyses on Spanish and English versions of the Progress in International Reading Literacy Study 2001 Reader booklet. Results confirmed that item cognitive loadings were a source of differential functioning favoring both England and the United States when compared against Colombia; however, adaptation effects did not consistently favor one group or the other. The second purpose of this study was to highlight strengths and limitations of Poly-SIBTEST for conducting substantive analyses of differential functioning sources and also to offer suggestions for future directions on this type of methodological research.

Author-supplied keywords

  • PIRLS
  • Poly-SIBTEST
  • international large-scale assessment
  • item cognitive loadings
  • sources of DIF
  • translation and adaptation effects

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Debra Sandilands

  • Maria Elena Oliveri

  • Bruno D. Zumbo

  • Kadriye Ercikan

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free