A Distributed Densely Connected Convolutional Network Approach for Enhanced Recognition of Health-Related Topics: A Societal Analysis Case Study

Abstract

Melanoma, a lethal form of skin cancer, poses a significant risk to global health if not detected and treated promptly. Its early detection is pivotal in increasing the likelihood of successful treatment and patient survival. However, the accurate diagnosis of melanoma remains a challenge, even for seasoned dermatologists. Consequently, there has been a growing interest in leveraging Machine Learning (ML) algorithms to augment the accuracy of melanoma diagnosis. Typically, melanoma is identified through dermoscopic imaging. Numerous previous studies have proposed the automated analysis of skin lesions using both traditional classification techniques and deep learning models. These analyses often involve the feeding of designed functions into traditional categorization systems. Nonetheless, the high visual similarity between different skin lesion types and the complexity of skin diseases often renders manual features insufficiently discriminative, leading to failure in various scenarios. Recent research suggests that convolutional networks with short connections between layers near the input and the output can be deeper, more precise, and more efficient in training. This paper adopts this approach and introduces the application of Hadoop's HdiDenseNet techniques. DenseNets offer several notable advantages: they alleviate the vanishing-gradient problem, enhance feature propagation, encourage feature reuse, and substantially reduce the number of parameters. The performance of our proposed architecture is evaluated against four highly competitive benchmark object identification challenges using a dataset comprising over 40,000 images sourced diversely. The results demonstrate that the most effective method is a densely connected distributed convolutional network, particularly when applied to patient metadata. Ultimately, this paper aims to contribute to the field of medical image analysis and potentially enhance the accuracy of melanoma diagnosis. By doing so, it could play a crucial role in improving patient prognosis and saving lives.