Investigation and Imitation of Human Captains' Maneuver Using Inverse Reinforcement Learning

Abstract

Automatic collision avoidance is of significant importance to prevent maritime collisions. Although many studies have been conducted in recent years, autonomous system has not completely replaced human captains since it is still difficult to imitate their complicated decisions. Thus, the present paper tries to investigate and imitate experienced captains' maneuver using maximum entropy inverse reinforcement learning (MaxEnt IRL). We firstly verify that MaxEnt IRL can reproduce appropriate reward function from demonstrative trajectories. Afterwards, we conduct an experiment on a simulator where well-experienced captains maneuver in congested sea and estimate reward from the trajectories. Searching the route which maximizes the obtained reward, finally, we demonstrate the optimized route can avoid collision against multiple ships in compliance with the International Regulations for Preventing Collisions at Sea (COLREGs).