Exam blueprinting as a tool to overcome principal validity threats: A scoping review

Abstract

Medical education's teaching and learning process evolves. This requires a change in instruction and assessment. An assessment's validity is threatened by poorly aligned tests or those with irrelevant variance. A well-designed test blueprint bridges gaps and integrates various education pillars, ensuring an accurate representation of learning outcomes and domains. This study reviewed the test blueprint approach in the relevant literature and highlighted its advantages in improving test measures, particularly those for validity and reliability. We searched MEDLINE/PubMed and Scopus for relevant articles; duplicates were eliminated, and those that met our eligibility criteria were chosen. For data extraction, a charting framework was created. Quantitative and qualitative data were reported and analyzed. Thematic analysis was performed for selected studies. To verify the selected studies, experts were consulted. Out of 487 selected studies, 22 were included in the review, 18 of which focused on blueprinting design and implementation, and 4 of which used blueprinting to improve medical curriculum design and evaluation practices. The review comprised a qualitative study, 2 cohort studies, 8 cross-sectional studies, 6 quasi-experimental studies, 3 review articles, one of which was a practical guide for test blueprint design, and 1 case report. Finally, as evidenced by their findings, the majority of studies addressed the qualities of a good test, the primary threats to validity and reliability, the benefits, and the methodologies for designing test blueprints. Nonetheless, a lack of experimental studies was observed. Electronic blueprinting and a blueprint for programmatic assessment approaches were lacking and warrant additional research.