Graph convolutional reinforcement learning for resource allocation in hybrid overlay–underlay cognitive radio network with network slicing

Abstract

Nowadays, wireless communication system is facing the problems of spectrum resource shortage. Cognitive radio technology allows cognitive users to use the spectrums authorized to primary users to improve the spectrum utilization. In this paper, a cognitive network model based on hybrid overlay–underlay spectrum access mode is established. To solve the resource allocation problem, a multi-agent resource allocation algorithm based on graph convolution reinforcement learning which combines deep Q network (DQN) and graph attention network is proposed. DQN is used for action selection and graph attention network is used to obtain the information about neighbours, so as to achieve local cooperation. The proposed algorithm can adaptively optimize cognitive network throughput, spectrum efficiency, or power efficiency by controlling the transmission power and channel selection of cognitive users. To improve the information interaction efficiency, the agent's states are divided into two categories, whether it needs to interact with neighbours or not, which shortens training time and improves convergence speed. Simulation results show that the proposed algorithm can effectively improve the power efficiency of cognitive networks. Compared with Q-learning, DQN and exiting graph convolutional reinforcement learning algorithm, the proposed algorithm has faster convergence speed and higher stability, and obtains higher network power efficiency.