Effective 3C Resource Utilization and Fair Allocation Strategy for Multi-Task Federated Learning

Abstract

Nowadays, one of the main challenges in expanding AI applications is the effective use of Computation, Communication, and Caching (3C) resources. The complexity of the cloud environment and the diversity of resource usage make it challenging to complete federated learning tasks in a cost-effective, timely, and seamless manner. To address these issues, this paper proposes a comprehensive approach to optimize the overall service efficiency of federated learning in time-varying and 3C-constrained environments. Firstly, a utility function based on convergence efficiency is proposed to reflect the physical benefits of processing AI tasks. Then, a fair allocation strategy consistent with the optimization goal is designed by modeling the task allocation process through virtual queue Lyapunov drift. Next, a Federated Learning Long Short-Term Memory (LSTM) based Queuing Optimization and Allocation Policy Calculation Algorithm (FL-QAPC) is proposed for resource allocation policy calculation using multi-dimensional network state inputs with time series. This algorithm implements predictive control based on historical records. Finally, a feasible experimental test platform is conducted, which is extended to an actual wireless mobile scenario based on 5G. The superiority of the proposed solution is verified through comparison with other benchmarks.