Deep reinforcement learning-based computation offloading with custom Q value function and loss function for mobile edge computing

Abstract

In the current cloud-edge-local environment, the optimization technology for resource allocation and task offloading decisions for multi-task processing is not yet mature, which may lead to resource waste and latency. To address this issue, this paper transforms the task offloading and resource allocation problem into a Markov Decision Process (MDP) and designs a dedicated state and action space to represent different offloading and resource allocation schemes. Given the characteristics of continuous action spaces, traditional Q-learning algorithms are no longer applicable. In response to this challenge, this paper redesigns the Q-value formula and employs a logical interpreter to optimize the numerical output of the neural network to solve for the optimal task offloading strategy and resource allocation scheme. Furthermore, this paper also studies the rules of changing neural network structures, neuron numbers, and the impact of task numbers on the convergence of the loss function. By comparing the optimal energy consumption of different offloading strategies, the offloading strategy and resource allocation scheme proposed in this paper achieves significantly lower energy consumption than other strategies and schemes, while considering the maximum delay limit, it saves about 55% compared to the random strategy, providing a new solution for multi-task processing in cloud-edge-local environments.