A BAYESIAN APPROACH BASED ON ACQUISITION FUNCTION FOR OPTIMAL SELECTION OF DEEP LEARNING HYPERPARAMETERS: A CASE STUDY WITH ENERGY MANAGEMENT DATA

Abstract

With the recent rollout of smart meters, huge amount of data can be generated on hourly and daily basis. Researchers and industry persons can leverage from this big data to make intelligent decisions via deep learning (DL) algorithms. However, the performance of DL algorithms are heavily dependent on the proper selection of parameters. If the hyperparameters are poorly selected, they usually lead to suboptimal results. Traditional approaches include a manual setting of parameters by trial and error methods which is time consuming and difficult process.  In this paper, a Bayesian approach based on acquisition is presented to automatic selection of optimal parameters based on provided data. The acquisition function was established to search for the best parameter from the input space and evaluate the next points based on past observations. The tuning process identifies the best model parameters by iterating the objective function and minimizing the loss for optimizable variables such as learning rate and Hidden layersize. To validate the presented approach, we conducted a case study on real-life energy management datasets while constructing a deep learning model on MATLAB platform. A performance comparison was drawn with random parameters and optimal parameters selected by presented approach. The comparison results illustrate that the presented approach is effective as it brings a notable improvement in the performance of learning algorithm.