Evaluation of optimal resource management policies for WiMAX networks with AMC: A reinforcement learning approach

Abstract

Call admission control in access network has become an interesting topic for the research community due to its potential applicability in broadband wireless systems. Admission control problem can be formulated as Markov decision process (MDP) and has proven to deliver optimal policies for blocking and dropping probabilities in wireless networks. This however typically requires the model to know the system dynamics in advance. One approach to solving MDPs considers letting CAC agent interact with the environment and learn by "trial and error" to choose optimal actions-thus Reinforcement Learning algorithms are applied. Abstraction and generalization techniques can be used with RL algorithms to solve MDPs with large state space. In this paper authors decribe and evaluate a MDP formulated problem to find optimal Call Admission Control policies for WiMAX networks with adaptive modulation and coding. We consider two classes of service (BE and UGS-priority) and a variable capacity channel with constant error bit rate. Hierarchical Reinforcement Learning (HRL) techniques are applied to find optimal policies for multi-task CAC agent. In addition this article validates several neural network training algorithms to deliver a training algorithm suitable for the CAC agent problem. © 2011 Springer-Verlag Berlin Heidelberg.