Emergent diagnoses from a collective of radiologists: Algorithmic versus social consensus strategies

Abstract

Twelve radiologists independently diagnosed 74 medical images. We use two approaches to combine their diagnoses: a collective algorithmic strategy and a social consensus strategy using swarm techniques. The algorithmic strategy uses weighted averages and a geometric approach to automatically produce an aggregate diagnosis. The social consensus strategy used visual cues to quickly impart the essence of the diagnoses to the radiologists as they produced an emergent diagnosis. Both strategies provide access to additional useful information from the original diagnoses. The mean number of correct diagnoses from the radiologists was 50 and the best was 60. The algorithmic strategy produced 63 correct diagnoses and the social consensus produced 67. The algorithm’s accuracy in distinguishing normal vs. abnormal patients (0.919) was significantly higher than the radiologists’ mean accuracy (0.861; p = 0.047). The social consensus’ accuracy (0.951; p = 0.007) showed further improvement.