Evaluating ChatGPT’s diagnostic accuracy in skin diseases based on images

Abstract

OBJECTIVES: This study aims to evaluate the performance of Dr. FerManda, a custom ChatGPT AI-based model, in diagnosing 42 common skin diseases. The focus is on assessing its diagnostic accuracy and the potential for AI-assisted dermatology without relying on detailed patient information like age, sex, or symptoms. METHODS: The Dr. FerManda model was trained using publicly available image datasets and clinical literature related to dermatological conditions. Its diagnostic accuracy was tested across various skin diseases and compared against evaluations by expert dermatologists. Additionally, the model provided descriptions of symptoms, causes, and treatment options for each diagnosed condition. RESULTS: The model achieved 100% accuracy across test cases, although it initially misdiagnosed two diseases; these errors were corrected following further guidance. It also delivered detailed and accurate information on each condition, aligning closely with expert dermatologists' assessments regarding symptoms, causes, and treatment recommendations. CONCLUSIONS: These findings indicate that AI, particularly custom ChatGPT models like Dr. FerManda, holds great promise for improving dermatological diagnostics. With its high accuracy and rapid response times, AI could significantly enhance diagnostic support in dermatology, paving the way for broader applications and future research to expand its capabilities.