Heuristic optimisation of multi-task dynamic architecture neural network (DAN2)

Abstract

This article proposes a novel method to optimise the Dynamic Architecture Neural Network (DAN2) adapted for a multi-task learning problem. The multi-task learning neural network adopts a multi-head and serial architecture with DAN2 layers acting as the basic subroutine. Adopting a dynamic architecture, the layers are added consecutively starting from a minimal initial structure. The optimisation method adopts an iterative heuristic scheme that sequentially optimises the shared layers and the task-specific layers until the solver converges to a small tolerance. Application of the method has demonstrated the applicability of the algorithm to simulated datasets. Comparable results to Artificial Neural Networks (ANNs) have been obtained in terms of accuracy and speed.