Improving computational robustness in log-likelihood maximization for binary outcomes

Abstract

A generalized linear model is a standard framework for assessing the impact of variables on an outcome. A typical approach for Maximum Likelihood Estimation (MLE) of its paramaters is by optimizing a log-likelihood function. The algorithms in standard software packages to solve these nonlinear equations are well-known to sometimes diverge, one  reason for which is certain expressions which blow up if the probability of success is close to 0 or 1. This article develops a Singularity Abating Maximum Likelihood Estimator (SAMLE) approach which nullifies these singularities by rewriting the problematic expressions. SAMLE's robustness on synthetic data, and on a readily available biostatistical data set, is illustrated through the development of a Newton–Raphson algorithm. We demonstrate that those situations also experience divergence in standard R implementations, indicating that a SAMLE rewriting is an easy modification which should be incorporated into MLE numerical algorithms with similar propensity for divergence.