An Online Test for Goodness-of-Fit in Logistic Regression Model

Abstract

In past decades, the goodness-of-fit test has been widely used to evaluate the calibration of prediction models. The test helps to determine whether poor predictions (lack of fit) are significant, which would indicate that problems exist in the model. However, the goodness-of-fit test is usually performed at the end of data collection, which may not detect changes in the model's fit as data are generated sequentially. In this paper, we examined the potential for using a new online goodness-of-fit test to determine the goodness-of-fit at each time point and provide an early signal if significant changes occur during model fitting. The simulation results indicate that the proposed online chi-square test was more sensitive than the traditional goodness-of-fit tests and online Hosmer-Lemeshow test for studies that aim to monitor the adequacy of a fitted model. An example using real hospital data is then used to illustrate the applicability of the proposed test.