A Bayesian perspective on constructing a written assessment of probabilistic clinical reasoning in experienced clinicians

Abstract

Rationale: Decision-making performance assessments have proven problematic for assessing clinical reasoning. Aims and Objectives: A Bayesian approach to designing an advanced clinical reasoning assessment is well grounded in mathematical and cognitive theory and may offer significant psychometric advantages. Probabilistic logic plays an important role in medical problem solving, and performances on Bayesian-type tasks appear to be causally-related to the ability to make sound clinical decisions. Methods: A validity argument is used to guide the design of an assessment of medical reasoning using clinical probabilities. Results/Conclusions: The practical advantage of using a Bayesian approach to item design relates to the fact that probability theory provides a rationally optimal method for managing uncertain information and provides the criteria for objective correct answer scoring. Potential item formats are discussed.