Nonlinear Regression Analysis

Abstract

Nonlinear regression analysis is a very popular technique in mathematical and social sciences as well as in engineering. In this article, we offer an introduction of theories and methods of nonlinear regression. Least-squares with Gauss–Newton method is the most widely used approach to parameters estimation. Under the normality assumption of errors, the least-squares estimates equal the maximum likelihood estimates. The predicted values of the responses can be biased because of the intrinsic nonlinearity of the model. Even if the degree of intrinsic nonlinearity is slight, the least-squares estimates of the parameters may still be hard to converge due to the parameter-effects nonlinearity. The intrinsic nonlinearity is invariant to reparametrization, while the parameter-effects nonlinearity can be corrected by a suitable reparametrization. We also discuss techniques from geometric viewpoints for estimation and inferences as well as for assessing their statistical properties.