Adaption of stepsize parameter using Newton's method

Abstract

A method to optimize stepsize parameters in exponential moving average (EMA) based on Newton's method to minimize square errors is proposed. The stepsize parameters used in reinforcement learning methods should be selected and adjusted carefully for dynamic and non-stationary environments. To find the suitable values for the stepsize parameters through learning, a framework to acquire higher-order derivatives of learning values by the stepsize parameters has been proposed. Based on this framework, the authors extend a method to determine the best stepsize using Newton's method to minimize EMA of square error of learning. The method is confirmed by mathematical theories and by results of experiments. © 2011 Springer-Verlag.