PSbBO-Net: A Hybrid Particle Swarm and Bayesian Optimization-based DenseNet for Lung Cancer Detection using Histopathological and CT Images

Abstract

Lung cancer remains a substantial global fatality; early detection is imperative for successful intervention and treatment. Deep learning (DL) models have shown promise in predicting lung cancer from medical images, but optimizing their parameters remains a challenging task. To improve prediction capability, this study introduces an approach by merging Particle Swarm Optimization and Bayesian Optimization (PSbBO) to optimize deep learning parameters. PSO provides an effective way for exploring the hyperparameter space, while Bayesian optimization provides a probabilistic framework for the effective evaluation and refining of a DL network. The simulation study showcases the effectiveness of the proposed model, achieving notable metrics for histopathological images, including an accuracy of 99.5%, precision of 98.3%, recall of 99.2%, F1-score of 99.4%, and an error rate of 1.19%. Furthermore, when applied to lung CT images, the proposed PSbBO demonstrates an accuracy of 98.8%, precision of 97.4%, recall of 98.3%, F1-score of 98.6%, and an error rate of 1.21%.