Factor analysis methods and validity evidence: a review of instrument development across the medical education continuum

  • Wetzel A
  • 4

    Readers

    Mendeley users who have this article in their library.
  • N/A

    Citations

    Citations of this article.

Abstract

PURPOSE: Instrument development consistent with best practices is necessary for effective assessment and evaluation of learners and programs across the medical education continuum. The author explored the extent to which current factor analytic methods and other techniques for establishing validity are consistent with best practices. METHOD: The author conducted electronic and hand searches of the English-language medical education literature published January 2006 through December 2010. To describe and assess current practices, she systematically abstracted reliability and validity evidence as well as factor analysis methods, data analysis, and reported evidence from instrument development articles reporting the application of exploratory factor analysis and principal component analysis. RESULTS: Sixty-two articles met eligibility criteria. They described 64 instruments and 95 factor analyses. Most studies provided at least one source of evidence based on test content. Almost all reported internal consistency, providing evidence based on internal structure. Evidence based on response process and relationships with other variables was reported less often, and evidence based on consequences of testing was not identified. Factor analysis findings suggest common method selection errors and critical omissions in reporting. CONCLUSIONS: Given the limited reliability and validity evidence provided for the reviewed instruments, educators should carefully consider the available supporting evidence before adopting and applying published instruments. Researchers should design for, test, and report additional evidence to strengthen the argument for reliability and validity of these measures for research and practice.

Author-supplied keywords

  • Education, Medical/ standards/ statistics & numeri
  • Educational Measurement/ standards/ statistics & n
  • Evidence-Based Medicine
  • Factor Analysis, Statistical
  • Humans
  • Practice Guidelines as Topic
  • Principal Component Analysis

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Authors

  • A P Wetzel

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free