Autonomous Decentralized Traffic Control Using Q-Learning in LPWAN

Abstract

Owing to the recent research and development on the Internet-of-Things (IoT) and machine-to-machine (M2M) communication, wireless sensor networks have attracted considerable attention. Among these networks, low power wide area networks (LPWANs), which realize low power, low data rate, and wide communication area, are most commonly used for long-range communication. These networks adopt asynchronous random-access protocols, such as the pure ALOHA protocol in the medium access control (MAC) layer. Thus, there is a high possibility that multiple nodes transmit packets simultaneously on the same frequency channel, resulting in packet collisions. Carrier-sense multiple access/collision avoidance (CSMA/CA) and centralized resource allocation are effective for avoiding packet collisions. However, these schemes increase the energy consumption of battery-powered LPWAN nodes. In addition, LPWAN has a large coverage area; hence, there is a high possibility that the carrier sense will not work successfully. Thus, this paper proposes a simple but effective machine-learning-based scheme that tackles the packet collision problem by offsetting the transmission timings and avoiding unnecessary packet transmission in an autonomous decentralized manner. Each LPWAN node adjusts the transmission probability and timing using the Q-learning technique. The proposed scheme provides effective packet collision avoidance for LPWAN nodes without the need for an additional control signal. The computer simulation results show that the proposed scheme can improve the average packet delivery ratio (PDR) by 60% compared to the pure ALOHA protocol.