Adaptive radio access technology selection on ESP32-based IoT devices using reinforcement learning

Abstract

The rapid proliferation of Internet of Things (IoT) applications has led to the coexistence of multiple wireless connectivity technologies within the same environment. In this context, the ability to integrate various Radio Access Technologies (RATs) simultaneously is essential for enhancing system adaptability. In this paper, we implement a multi-RAT system on a resource-constrained ESP32 module. The system combines Wi-Fi, Bluetooth Low Energy (BLE), and cellular networks to provide flexible and resilient IoT connectivity. However, with multiple RATs available, the challenge of selecting the most appropriate RAT remains critical. To overcome this issue, we propose a multi-criteria adaptive decision algorithm based on Naive Reinforcement Learning (NRL). This solution dynamically selects the optimal RAT based on current network conditions and application requirements, while respecting the limited memory and processing capacities of ESP32-based devices. The practicality and effectiveness of the proposed NRL-based algorithm are demonstrated through an e-health scenario using real-world healthcare data. This demonstrates a significant increase in the performance of key metrics. The method increases Packet Delivery Ratio (PDR) as compared to using a single RAT and randomizing the selection of the RAT. It also reduces the average end-to-end time and energy consumption compared to the single RAT method and selecting RAT at random.