Evaluation of a smartphone application for diagnosis of skin diseases

Abstract

Introduction: Artificial intelligence (AI) could offer equal, or even more accurate, diagnoses of melanoma than most dermatologists. However, the value of popular smartphone applications for diagnosing unpigmented skin lesions remains unclear. Aim: To compare the diagnostic accuracy of a popular, free-to-use web application for automatic dermatosis diagnosis against expert diagnosis of selected skin diseases. Material and methods: Skin lesion images of patients with verified diagnosis were collected using a smartphone and were diagnosed by the application. The AI provided five diagnoses of varying probability. For each patient, accuracy of the diagnosis was evaluated by three criteria, i.e. whether the expert diagnosis was matched by the most probable automated diagnosis, one of the top three diagnoses or one of the top five diagnoses. Reliability was analysed using intraclass correlation coefficients. Results: The chance of a correct diagnosis increased when more outcomes were considered and more samples of a skin condition were included. However, the probability of a diagnosis repeating for the same patient was below 25%. Reliability, sensitivity and specificity were insufficient for clinical purposes. Conclusions: Although AI diagnostics are encouraging, there is also a large margin for improvement, and AI is not yet an adequate replacement for medical professionals.