Reinforcement Learning Generalization for Quadrotor With Slung Load Systems Through Homogeneity Transformations

Abstract

Load transportation through unmanned aerial vehicles (UAVs), such as quadrotors, has a high potential for quick deliveries to locations that are out of the reach of ground vehicles. The complexity of the pick-and-place procedure in such tasks increases if the target location does not have a clearance at the top, necessitating the use of recent learning-based controllers such as reinforcement learning (RL). This article presents a new concept of dual-scale homogeneity, a property defined by scaled magnitudes and time in transformed coordinates that remain independent of system parameters. It demonstrates that applying transformations to achieve this property ensures consistent performance of a quadrotor with a slung load system (QSLS) despite variations in its parameters. Furthermore, it also presents an effective approach to design a parameter-dependent RL policy that homogenizes the QSLS. Unlike plain RL or gain-scheduled proportional-integral-derivative controllers, which confine parameter variations within a predefined range encountered during training or tuning, the developed approach works under large parameter variations, significantly surpassing the performance of traditional controllers. The conducted experiments on load placement in a confined space, utilizing a quadrotor to manage load swing, proved the proposed synergy between the homogeneity transformations and RL, yielding a success rate of 96% in bringing the load to its designated target with a 3-D RMSE of 0.0253 m.