A hybrid deep Q-network for the SVM Lagrangian

Abstract

The setting hyperparameters in the support vector machine (SVM) is very important with regard to its accuracy and efficiency. In this paper, we employ a novel definition of the reinforcement learning state, actions and reward function that allows a deep Q-network (DQN) to learn to control the optimization hyperparameters for the SVM deep neural networks by supervised Big-Data. In this framework, the DQN algorithm with experience replay is based on the off-policy reinforcement learning for the expected discounted return of rewards, or q-values, connected to the actions of adjusting the hyperprameters in the SVM. We propose the two deep neural networks, one with the SVM and the other with Q-network (DQN). The SVM deep neural networks learns a policy for the optimization hyperparameters, but differ in the number of allowed actions. The SVM deep neural networks trains the hyperparameters of the SVM simultaneously such as the Lagrangian multiplier. The proposed algorithm is called a Hybrid DQN combined with SVM deep neural networks. This algorithm could be considered as the classifier in the real-world domains such as network anomalies in the distributed server loads, because the SVM is suitable for the application in a classification, especially for the one-against-the others. Algorithm comparisons show that our proposed algorithm leads to good optimization of the Lagrangian multiplier and can prevent overfitting to a certain extent automatically without human system designers. In terms of the classification performance of the proposed algorithm can be compared to the original LIBSVM with no controls of the hyperparameters.