Learning parameters in deep belief networks through firefly algorithm

Abstract

Restricted Boltzmann Machines (RBMs) are among the most widely pursed techniques in the context of deep learning-based applications. Their usage enables sundry parallel implementations, which have become pivotal in nowadays large-scale-oriented applications. In this paper, we propose to address the main shortcoming of such models, i.e. how to properly fine-tune their parameters, by means of the Firefly Algorithm, as well as we also consider Deep Belief Networks, a stacked-driven version of the RBMs. Additionally, we also take into account Harmony Search, Improved Harmony Search and the well-known Particle Swarm Optimization for comparison purposes. The results obtained showed the Firefly Algorithm is suitable to the context addressed in this paper, since it obtained the best results in all datasets.