Entropy and PCA Analysis for Environments Associated to Q-Learning for Path Finding

Abstract

This work is based on the simulation of the reinforcement learning method for path search for mobile robotic agents in unstructured environments. The choice of the learning and reward coefficients of the Q-learning method affect the number of average actions that the algorithm requires for reach the goal from the start position. In addition, another important factor is the randomness degree and environment size over which the path must be calculated, since they affect the time of convergence in the learning. Likewise, a performance metric of the Q-learning algorithm is proposed, based on the Entropy and Principal Component Analysis of the environment representative images. The analysis by Entropy only allows to determine, in a scalar form, the environment randomness degree, but it does not provide information about the dispersion location. In contrast, the analysis by PCA allows to quantify not only the randomness, but also helps to estimate the direction of greater randomness of the environment. The advantage of this analysis by PCA and Entropy is that one could estimate the actions number or movements required for path search algorithms based on the randomness of unstructured environments.