Retrieval Augmented Medical Diagnosis System

Abstract

Subjective variability in human interpretation of diagnostic imaging presents significant clinical limitations, potentially resulting in diagnostic errors and increased healthcare costs. While artificial intelligence (AI) algorithms offer promising solutions to reduce interpreter subjectivity, they frequently demonstrate poor generalizability across different healthcare settings. To address these issues, we introduce Retrieval Augmented Medical Diagnosis System (RAMDS), which integrates an AI classification model with a similar image model. This approach retrieves historical cases and their diagnoses to provide context for the AI predictions. By weighing similar image diagnoses alongside AI predictions, RAMDS produces a final weighted prediction, aiding physicians in understanding the diagnosis process. Moreover, RAMDS does not require complete retraining when applied to new datasets; rather, it simply necessitates re-calibration of the weighing system. When RAMDS fine-tuned for negative predictive value was evaluated on breast ultrasounds for cancer classification, RAMDS improved sensitivity by 21% and negative predictive value by 9% compared to ResNet-34. Offering enhanced metrics, explainability, and adaptability, RAMDS represents a notable advancement in medical AI. RAMDS is a new approach in medical AI that has the potential for pan-pathological uses, though further research is needed to optimize its performance and integrate multimodal data.