Convergence Rate of Sieve Estimates

Abstract

In this paper, we develop a general theory for the convergence rateof sieve estimates, maximum likelihood estimates (MLE's) and relatedestimates obtained by optimizing certain empirical criteria in generalparameter spaces. In many cases, especially when the parameter spaceis infinite dimensional, maximization over the whole parameter spaceis undesirable. In such cases, one has to perform maximization overan approximating space (sieve) of the original parameter space andallow the size of the approximating space to grow as the sample sizeincreases. This method is called the method of sieves. In the caseof the maximum likelihood estimation, an MLE based on a sieve iscalled a sieve MLE. We found that the convergence rate of a sieveestimate is governed by (a) the local expected values, variancesand L_2 entropy of the criterion differences and (b) the approximationerror of the sieve. A robust nonparametric regression problem, amixture problem and a nonparametric regression problem are discussedas illustrations of the theory. We also found that when the underlyingspace is too large, the estimate based on optimizing over the wholeparameter space may not achieve the best possible rates of convergence,whereas the sieve estimate typically does not suffer from this difficulty.