Increasing Prevalence Expectation in Thoracic Radiology Leads to Overcall

Abstract

Rationale and Objectives: The aim of this study was to measure the effect of prevalence expectation as determined by clinical history on the diagnostic performance of radiologists during pulmonary nodule detection on adult chest radiographs. Materials and Methods: A multi-observer, counter-balanced study (having half the readers in each group read a different condition initially) was performed to assess the effect of abnormality expectation on experienced radiologists' performance. A total of 33 board-certified radiologists were divided into three groups and searched for evidence of malignancy on a single set of 47 postero-anterior (PA) chest radiographs, 10 of which contained a single pulmonary nodule. The radiologists were unaware of disease prevalence. Before each viewing of the same dataset, the radiologists were allocated to two of three conditions based on the differing clinical information (previous cancer, no history, visa applicant). Location sensitivity, specificity, and jack-knife free-response receiver operator characteristics figure of merit were used to compare radiologist performance between conditions. Results: A significant reduction in specificity was shown for the cancer compared to that for the visa condition (W = -41 P = 0.02). No other significant findings were demonstrated for this or the other condition comparisons. No significant difference in the performance of radiologists was noted when viewing images under the same conditions. Conclusions: This study suggested that there is a reduction in specificity with high compared to low prevalence expectation following specific radiological contexts. A reduction in specificity can have important clinical consequences leading to unnecessary interventions. The results and their implications emphasize the caution that should be placed on providing accurate referral criteria.