Path Planning of Robotic Fish in Unknown Environment with Improved Reinforcement Learning Algorithm

Abstract

Path planning is the primary task for robotic fish, especially when the environment under water of robotic fish is unknown. The conventional reinforcement learning algorithms usually exhibit a poor convergence property in unknown environment. In order to find the optimal path and increase the convergence speed in the unknown environment, an improved reinforcement learning method utilizing a simulated annealing approach is proposed in robotic fish navigation. The simulated annealing policy with a novel cooling method rather than a general ɛ-greedy policy is taken for action choice. The algorithm convergence speed is improved by a novel reward function with goal-oriented strategy. Then the stopping condition of the proposed reinforcement learning algorithm is rectified as well. In this work, the robotic fish is designed and the prototype is produced by 3D printing technology. Then the proposed algorithm is examined in the 2D unpredictable environment to obtain greedy actions. Experimental results show that the proposed algorithms can generate an optimal path in unknown environment for robotic fish and increase the convergence speed as well as balance the exploration and exploitation.