Value of clinical review for AI-guided deep vein thrombosis diagnosis with ultrasound imaging by non-expert operators

Abstract

Deep vein thrombosis (DVT) carries high morbidity, mortality, and costs globally. Point of care ultrasound (POCUS) image acquisition by non-ultrasound-trained providers, supported by an AI-based guidance and remote image review system, is believed to improve the timeliness and cost-effectiveness of diagnosis. We examine a database of 381 patients with suspected DVT who underwent an AI-guided ultrasound scan by a non-ultrasound-trained nurse and an expert sonographer-performed standard compression ultrasound scan. Each AI-guided scan was reviewed remotely by blinded radiologists or blinded independent POCUS-certified American Emergency Medicine (EM) physicians. Remote reviewer and standard scan diagnoses were compared. The primary endpoint is AI-guidance system sensitivity with clinician review; secondary endpoints include specificity, positive predictive value, negative predictive value, image quality, inter-observer image quality, and vein compressibility agreement. Data was analysed through the bootstrapping method, bootstrapping with a second reader for each scan, and a majority voting system. Eighty percent (n = 304) of scans were of sufficient diagnostic quality. Radiologist reviewer sensitivity ranged from 90%–95%, specificity from 74%–84%, NPV from 98%–99%, PPV from 30%–42%, and potential expert-led ultrasound scans avoided from 39%–50%. Inter-observer agreement for image quality was 0.15 and for compressibility 0.61. EM reviewer sensitivity ranged from 95%–98%, specificity from 97%–100%, NPV was 99%, PPV from 81%–100%, and potential expert-led ultrasound scans avoided from 29%–38%. Inter-observer agreement for image quality was 0.59 and for compressibility 0.67. Diagnosing lower extremity DVT through AI-guided image acquisition with clinician review is feasible. Performance is influenced by reviewer expertise. We find potential positive impacts on health economics, including safely avoiding expert-led ultrasound scans.