Evaluation and Analysis of Performances of Different Heuristics for Optimal Tuning Learning on Mamdani Based Neuro-Fuzzy System

Abstract

The optimization algorithms based on gradients have a series of hyperparameters that will allow among other things to reach a global minimum of the error function, convergence, and at an adequate velocity, trying to avoid overtraining, divergent solutions and overload of computation to execute an exaggerated number of iterations for its process, hence, greater attention is paid to the learning rate as the main parameter to be adjusted during training to achieve a better performance in learning, for this reason, the contribution of this paper is to show initially the positive and negative impact that would have to use a suitable or not learning rate, the benefits of its adjustment during the learning phase, as well as the evaluation and analysis of the performance of different heuristics implemented on a gradient-based algorithm with momentum and adaptive learning rate, with the purpose of being able to have knowledge of their performance in different contexts and scenarios, since we know that this parameter will depend greatly on the complexity of the data to be treated, so that a fixed number cannot be determined for all cases, but if you can try to identify different scenarios, which can be managed from different heuristics for the optimal adjustment of the parameter during its learning process.