Asymptotic Behavior of $M$-Estimators of $p$ Regression Parameters when $p^2/n$ is Large. I. Consistency

Abstract

Consider the general linear model Y = xβ + R with Y and R n-dimensional, β p-dimensional, and X an n × p matrix with rows x'i. Let ψ be given and let $\hat\beta$ be an M-estimator of β satisfying $0 = \sum x_i\psi(Y_i - x'_i\hat\beta)$ . Previous authors have considered consistency and asymptotic normality of $\hat\beta$ when p is permitted to grow, but they have required at least p2/n → 0. Here the following result is presented: in typical regression cases, under reasonable conditions if p(log p)/n → 0 then |β̂ - β|2 = Op(p/n). A subsequent paper will show that β̂ has a normal approximation in Rp if (p log p)3/2/n → 0 and that maxi|x'i(β̂ - β)| →p 0 (which would not follow from norm consistency if p2/n → ∞). In ANOVA cases, β̂ is not norm consistent, but it is shown here that max|x'i(β̂ - β)| →p 0 if p log p/n → 0. A normality result for arbitrary linear combinations a'(β̂ - β) is also presented in this case.