L2 boosting in kernel regression

Abstract

In this paper, we investigate the theoretical and empirical properties of L2 boosting with kernel regression estimates as weak learners. We show that each step of L2 boosting reduces the bias of the estimate by two orders of magnitude, while it does not deteriorate the order of the variance. We illustrate the theoretical findings by some simulated examples. Also, we demonstrate that L2 boosting is superior to the use of higher-order kernels, which is a well-known method of reducing the bias of the kernel estimate. © 2009 ISI/BS.