Min-Max Bias Robust Regression

Abstract

This paper considers the problem of minimizing the maximum asymptotic bias of regression estimates over ε-contamination neighborhoods for the joint distribution of the response and carriers. Two classes of estimates are treated: (i) M-estimates with bounded function ρ applied to the scaled residuals, using a very general class of scale estimates, and (ii) bounded influence function type generalized M-estimates. Estimates in the first class are obtained as the solution of a minimization problem, while estimates in the second class are specified by an estimating equation. The first class of M-estimates is sufficiently general to include both Huber Proposal 2 simultaneous estimates of regression coefficients and residuals scale, and Rousseeuw-Yohai S-estimates of regression. It is shown than an S-estimate based on a jump-function type ρ solves the min-max bias problem for the class of M-estimates with very general scale. This estimate is obtained by the minimization of the α-quantile of the squared residuals, where α = α(ε) depends on the fraction of contamination ε. When ε → 0.5, α(ε) → 0.5 and the min-max estimator approaches the least median of squared residuals estimator introduced by Rousseeuw. For the bounded influence class of GM-estimates, it is shown the "sign" type nonlinearity yields the min-max estimate. This estimate coincides with the minimum gross-error sensitivity GM-estimate. For p = 1, the optimal GM-estimate is optimal among the class of all equivariant regression estimates. The min-max S-estimator has a breakdown point which is independent of the number of carriers p and tends to 0.5 as ε increases to 0.5, but has a slow rate of convergence. The min-max GM-estimate has the usual rate of convergence, but a breakdown point which decreases to zero with increasing p. Finally, we compare the min-max biases for both types of estimates, for the case where the nominal model is multivariate normal.