Bridging intentions and outcomes: a program theory approach to residency accreditation standards

Abstract

Background: Accreditation systems can significantly influence medical graduates' competency and healthcare delivery through their impact on multiple stages of physician training. While substantial progress has been made in medical school accreditation systems globally, residency (Postgraduate Medical Education) accreditation faces additional challenges. The widespread transition to competency-based education and upcoming international recognition initiatives makes systematic evaluation of residency accreditation particularly timely. This study examined the clarity, comprehensiveness, adequacy, and validity of current Israeli residency accreditation standards as a case example for evaluating such systems. Methods: We conducted a qualitative evaluation study using Program Theory framework to analyse multiple data sources: Supreme Accreditation Board protocols (2014–2023), documents, including standards and questionnaires from 11 purposively sampled specialties, and semi-structured interviews with 28 stakeholders. Interviews were conducted in three rounds (2021–2024), with participants selected to represent diverse perspectives including decision-makers, department heads, scientific societies, residents, and site visit teams. Analysis focused on constructing outcome chains and developing outcome-action matrices, followed by critical examination of standards' structure, content, and validity. Results: Analysis revealed partial alignment between intended outcomes and implementation mechanisms. While most immediate outcomes relating to training programs received full or partial representation in standards, intermediate outcomes concerning resident development and final outcomes regarding graduate competencies often lacked explicit support. Standards showed varying degrees of clarity and consistency, potentially leading to divergent interpretations among stakeholders. The system demonstrated strong contextual awareness but faced tensions between medical service demands and ensuring quality, particularly regarding peripheral training sites. Several critical components were identified as missing from current standards, reflecting gaps in educational processes, oversight mechanisms, and organizational responsibilities. Conclusions: These findings identify specific areas requiring attention in residency accreditation standards while providing a structured approach for standards evaluation. Key recommendations include establishing clear objectives before standards development, making explicit choices between threshold and aspirational requirements, examining underlying assumptions, considering unwritten expectations, and structuring standards to optimize expectations alignment among stakeholders. This evaluation framework can support other accreditation bodies in reviewing and updating their standards while maintaining context-appropriate content.