Review and comparison of variable step-size LMS algorithms

Abstract

The inherent feature of the Least Mean Squares (LMS) algorithm is the step size, and it requires careful adjustment. Small step size, required for small excess mean square error, results in slow convergence. Large step size, needed for fast adaptation, may result in loss of stability. Therefore, many modifications of the LMS algorithm, where the step size changes during the adaptation process depending on some particular characteristics, were and are still being developed. The paper reviews seventeen of the best known variable step-size LMS (VS-LMS) algorithms to the degree of detail that allows to implement them. The performance of the algorithms is compared in three typical applications: parametric identification, line enhancement, and adaptive noise cancellation. The paper suggests also one general modification that can simplify the choice of the upper bound for the step size, which is a crucial parameter for many VS-LMS algorithms.