Assessing model performance: The Gini statistic and its standard error

Abstract

Database marketers frequently create statistical models to assess customers in terms of loyalty, lifetime value, responsiveness or acquisition. An assortment of modeling techniques, for example RFM (Recency-Frequency-Monetary value) models, regression, logit, neural networks and genetic algorithms, have been investigated to determine the most appropriate and effective technique. For the most part, traditional statistical measures utilize R 2, the F statistic, the Chi Square statistic, various classification indices and so forth to assess model performance- with an emphasis on goodness of fit, and measuring how closely data points fit a statistical model. Practitioners, on the other hand, typically use summarized descriptive methods to assess model performance: decile analysis, lift charts, cumulative lift charts, gains tables and cumulative gains tables. Both of these approaches have limitations. Some academic researchers have suggested that traditional goodness of fit statistics are not appropriate for evaluating model performance when the objective is to create models that maximize differentiation between population segments in terms of response rates. The traditional statistical measures are appropriate for assessing how well individual response values fit a given model (for example minimize least square errors between response data points and predicted values); however, they are not appropriate for effectively selecting market segments or individual customers for targeting and meeting business objectives. The descriptive measures used by practitioners, although visually appealing, do not assess overall model performance with statistical certainty. This research offers a remedy for the current situation by proposing the use of the Gini statistic and the associated standard error. We explain the Gini statistic and how it is connected to commonly used assessment measures. We then describe a simple method for computing Gini and its standard error. The accuracy of the method is demonstrated with specific industry data files. © 2010 Macmillan Publishers Ltd.