Parallelized Metaheuristic-Ensemble of Heterogeneous Feedforward Neural Networks for Regression Problems

Abstract

A feedforward neural network ensemble trained through metaheuristic algorithms has been proposed by researchers to produce a group of optimal neural networks. This method, however, has proven to be very time-consuming during the optimization process. To overcome this limitation, we propose a metaheuristic-based learning algorithm for building an ensemble system, resulting in shorter training time. In our proposed method, a master-slave based metaheuristic algorithm is employed in the optimization process to produce a group of heterogeneous feedforward neural networks, in which the global search operations are executed on the master, and the tasks of objective evaluation are distributed to the slaves (workers). To reduce evaluation costs, the entire training dataset is randomly divided equally into several disjoint subsets. Each subset is randomly paired with another subset of the remainder and distributed to a worker for the objective evaluation. Following the optimization process, representative candidate solutions (individuals) from the entire population are selected to perform as the base components of the ensemble system. The performance of the proposed method has been compared with those of other state-of-the-art techniques in over 31 benchmark regression datasets taken from public repositories. The experimental results show that the proposed method not only reduces the computational time but also achieves significantly better prediction accuracy. Moreover, the proposed method achieved promising results in the application of a subset of the million song dataset, which identifies the release year of a song and predicts the buzz on Twitter.