Multi-tier Collaborative Deep Reinforcement Learning for Non-terrestrial Network Empowered Vehicular Connections

Abstract

With the objective of supporting next generation driving services, non-terrestrial networks (NTNs) with low earth orbit (LEO) satellites have been regarded as promising paradigms to implement global ubiquitous and high-capacity vehicular connections. However, due to the high moving speed, different satellites can only service a specific set of vehicles for few minutes. In such case, due to the limited computing capability of the satellite, machine learning (ML) based and non-ML based solutions cannot be performed within such a short duration. To address these issues, in this paper, we propose a multi-tier collaborative deep reinforcement learning (DRL) scheme for resource allocation in NTN empowered vehicular networks, in which ground vehicles and LEO satellites maintain DRL-based decision model to obtain resource allocation decisions cooperatively. Specifically, ground vehicles with powerful computing capabilities can assist the satellite to tackle resource allocation optimizations, and the satellite determines final decisions and model parameters by aggregating local calculated results of vehicles. Additionally, the parameters of DRL-based decision model can be transferred from the current satellite to its successor as the starting point for future resource allocation decision-makings. Comprehensive simulations have been conducted to show the effectiveness of our proposed scheme.