Joint Channel Selection and Data Scheduling in HF Jamming Environment: An Interference-Aware Reinforcement Learning Approach

Abstract

In this paper, we study the joint problem of multichannel selection and data scheduling for high-frequency (HF) communication under jamming environment. Prior anti-jamming work mainly discussed fixed transmission time and considered saturated scenarios in which the agent always had packets to transmit. But HF dynamic spectrum environment and time-varying communication demand make traditional anti-jamming methods ineffective. To cope with above challenge, dynamic transmission time and packet scheduling are considered in this manuscript. The transmitter selects the working channel and the number of packets to be transmitted according to current jamming environment and buffer state. Simultaneously, channel diversity which allows the transmitter to select multiple channels to send same packet is also considered to overcome the unreliable and unstable characteristics of HF channel. We formulate the decision making process as a Markov decision process (MDP) and propose an interference-aware reinforcement learning algorithm. The proposed algorithm combining Q learning with upper confidence bounds for trees (UCT) balances exploration and exploitation of action set. Simulation results show that higher network throughput and less packet loss are obtained by the proposed algorithm.