Deep q-learning algorithm for solving inverse kinematics of four-link manipulator

Abstract

This paper presents deep Q-learning algorithm designed to solve inverse kinematics problem of four-link manipulator. This algorithm uses dynamic exploration coefficient instead of a constant value, which allows to avoid convergence of the neural network to a local optimum. In addition, a method for generating a Q-table has been developed to avoid the bottleneck effect when neural network construction. This in turn leads to reduction of training time and lower hardware requirements. To evaluate the effectiveness of the proposed algorithm, three environments were developed and for each of them specific neural networks model were used. Three different environments allow to evaluate the algorithm performance for solving inverse kinematics of varying complexity: with one initial and one target points, with several initial and one target points, and, conversely, with one initial and several target points. Obtained dependency graph of rewards on the number of training episodes shown successful training of agents in all environments. Successful training of the Q-learning algorithm in the third environment suggests that the algorithm can be used for solving the inverse kinematics for all points of the manipulator working space. The main advantage of the developed algorithm is the possibility of its application for solving inverse kinematics problems of varying complexity. In addition, this algorithm can be used to solve inverse kinematics of manipulator with a different number of links.