A hierarchical optimisation framework for pigmented lesion diagnosis

Abstract

The study of training hyperparameters optimisation problems remains underexplored in skin lesion research. This is the first report of using hierarchical optimisation to improve computational effort in a four-dimensional search space for the problem. The authors explore training parameters selection in optimising the learning process of a model to differentiate pigmented lesions characteristics. In the authors' demonstration, pretrained GoogleNet is fine-tuned with a full training set by varying hyperparameters, namely epoch, mini-batch value, initial learning rate, and gradient threshold. The iterative search of the optimal global-local solution is by using the derivative-based method. The authors used non-parametric one-way ANOVA to test whether the classification accuracies differed for the variation in the training parameters. The authors identified the mini-batch size and initial learning rate as parameters that significantly influence the model's learning capability. The authors' results showed that a small fraction of combinations (5%) from constrained global search space, in contrarily to 82% at the local level, can converge with early stopping conditions. The mean (standard deviation, SD) validation accuracies increased from 78.4 (4.44)% to 82.9 (1.8)% using the authors' system. The fine-tuned model's performance measures evaluated on a testing dataset showed classification accuracy, precision, sensitivity, and specificity of 85.3%, 75.6%, 64.4%, and 97.2%, respectively. The authors' system achieves an overall better diagnosis performance than four state-of-the-art approaches via an improved search of parameters for a good adaptation of the model to the authors' dataset. The extended experiments also showed its superior performance consistency across different deep networks, where the overall classification accuracy increased by 5% with this technique. This approach reduces the risk of search being trapped in a suboptimal solution, and its use may be expanded to network architecture optimisation for enhanced diagnostic performance.