Regularization and suboptimal solutions in learning from data

Abstract

Supervised learning from data is investigated from an optimization viewpoint. Ill-posedness issues of the learning problem are discussed and its Tikhonov, Ivanov, Phillips, and Miller regularizations are analyzed. Theoretical features of the optimization problems associated with these regularization techniques and their use in learning tasks are considered. Weight-decay learning is investigated, too. Exploiting properties of the functionals to be minimized in the various regularized problems, estimates are derived on the accuracy of suboptimal solutions formed by linear combinations of n-tuples of computational units, for values of n smaller than the number of data. © 2009 Springer-Verlag Berlin Heidelberg.