Automated Design of the Deep Neural Network Pipeline

Abstract

Deep neural networks have proven to be effective in various domains, especially in natural language processing and image processing. However, one of the challenges associated with using deep neural networks includes the long design time and expertise needed to apply these neural networks to a particular domain. The research presented in this paper investigates the automation of the design of the deep neural network pipeline to overcome this challenge. The deep learning pipeline includes identifying the preprocessing needed, the feature engineering technique, the neural network to use and the parameters for the neural network. A selection pertubative hyper-heuristic (SPHH) is used to automate the design pipeline. The study also examines the reusability of the generated pipeline. The effectiveness of transfer learning on the generated designs is also investigated. The proposed approach is evaluated for text processing—namely, sentiment analysis and spam detection—and image processing—namely, maize disease detection and oral lesion detection. The study revealed that the automated design of the deep neural network pipeline produces just as good, and in some cases better, performance compared to the manual design, with the automated design requiring less design time than the manual design. In the majority of instances, the design was not reusable; however, transfer learning achieved positive transfer of designs, with the performance being just as good or better than when transfer learning was not used.