Flexible unmanned surface vehicles control using probabilistic model-based reinforcement learning with hierarchical Gaussian distribution

Abstract

This paper focuses on improving the flexibility of probabilistic model-based reinforcement learning (MBRL) in unmanned surface vehicles (USV) against complicated ocean disturbances. Filtered probabilistic model predictive control using hierarchical Gaussian distribution (FPMPC-HG) is proposed to provide a more flexible representation of environmental uncertainties and better control capability against real-time disturbances by integrating hierarchical Gaussian distribution into existing approaches specific to USVs. The proposed approach was evaluated through position-keeping and targets-tracking tasks in a real boat data-driven simulation. The experimental results demonstrated significant improvement in control performance, generalization capability, and task completion compared to the baseline approaches without employing hierarchical Gaussian distribution. The improved expressivity and flexibility of the probabilistic model and the corresponding policy of USVs against unknown ocean disturbances indicate the potential of hierarchical Gaussian distribution in probabilistic MBRL as a growing trend in the USV domain.