Planet-wide performance of a skin disease AI algorithm validated in Korea

Abstract

To address the diversity of skin conditions and the low prevalence of skin cancers, we curated a large hospital dataset (National Information Society Agency, Seoul, Korea [NIA] dataset; 70 diseases, 152,443 images) and collected real-world webapp data (https://modelderm.com; 1,691,032 requests). We propose a conservative evaluation method by assessing sensitivity in hospitals and specificity in real-world use, assuming all malignancy predictions were false positives. Based on three differential diagnoses, skin cancer sensitivity in Korea was 78.2% (NIA) and specificity was 88.0% (webapp). Top-1 and Top-3 accuracies for 70 diseases (NIA) were 43.3% and 66.6%, respectively. Analysis of webapp data provides insights into disease prevalence and public interest across 228 countries. Malignancy predictions were highest in North America (2.6%) and lowest in Africa (0.9%), while benign tumors were most common in Asia (55.5%), and infectious diseases were most prevalent in Africa (17.1%). These findings suggest that AI can aid global dermatologic surveillance.