Learning Random Access Schemes for Massive Machine-Type Communication With MARL

Abstract

This paper investigates various multi-agent reinforcement learning (MARL) techniques for designing grant-free random access (RA) schemes suitable for low-complexity, low-power battery-operated devices in massive machine-type communication (mMTC). Previous studies on RA with MARL have shown limitations in terms of scalability and suitability for mMTC. To address scalability and practicality of the proposed methods, we examine the impact of excluding agent identification in the observation vector of each agent on network performance. We employ value decomposition networks (VDN) and QMIX algorithms with parameter sharing (PS) and compare their policies with the deep recurrent Q-network (DRQN). Our simulation results demonstrate that the MARL-based RA schemes can achieve a better throughput-fairness trade-off between agents without having to condition on the agent identifiers. We also present a correlated traffic model, which is more descriptive of mMTC scenarios, and show that the proposed algorithm can easily adapt to traffic non-stationarities. Moreover, the robustness of the proposed method in terms of scalability is also shown through simulations.