The parameter estimation of logistic regression with maximum likelihood method and score function modification

Abstract

The maximum likelihood parameter estimation method with Newton Raphson iteration is used in general to estimate the parameters of the logistic regression model. Parameter estimation using the maximum likelihood method cannot be used if the sample size and proportion of successful events are small, since the iteration process will not yield a convergent result. Therefore, the maximum likelihood method cannot be used to estimate the parameters. One way to resolve this un-convergence problem is using the score function modification. This modification is used to obtain the parameters estimate of logistic regression model. An example of parameter estimation, using maximum likelihood method with small sample size and proportion of successful events equals 0.1, showed that the iteration process is not convergent. This non-convergence can be solved with modifications on a score function. Modification on score function is to change a score function, a matrix of the first derivative of the log likelihood function, to the first derivative matrix itself minus multiplication of information matrix and biased vector. The modification of the score function can quickly yield values of parameter estimates, especially when the sample sizes are larger, and convergence was reached before the 10th iteration.