A Multi-input Architecture for the Classification of Skin Lesions Using ResNets and Metadata

Abstract

Skin illnesses are one of the most frequent diseases in the world, ranking fourth in terms of non-fatal human sickness, with an annual increase rate of 46.8% from 1990 to 2017. In this paper, we present a multi-input deep learning architecture for detecting and classifying Atopic Dermatitis, Papular Urticaria, and Scabies, three non-cancerous and common skin illnesses affecting children in Ethiopia. The suggested architecture comprises of a pre-trained ResNet architecture (ResNet101 and ResNet50) that has been fine-tuned on a dataset of 1796 photos and a convolutional neural network (CNN) that has been trained on tabular information associated with each image. We present a method for translating metadata to picture format by leveraging the correlation between each feature to establish their spatial and intensity values. On ResNet101, the architecture obtained average precision, recall, f1, and accuracy of 0.94, 0.94, 0.95, and 0.95, respectively, while on ResNet50, the architecture achieved average precision, recall, f1, and accuracy of 0.94, 0.92, 0.93, and 0.94, respectively. The lighter ResNet50 architecture was also integrated into an Android application.