Evaluating the Evaluation Metrics for Spatial Disease Cluster Detection Algorithms

Abstract

We show that the usual evaluation metrics used in machine learning are not appropriate to measure the performance of spatial disease cluster detection algorithms. We demonstrate that the usual recall and precision metrics give a distorted evaluation of the algorithms. To solve this problem, we propose new metrics based on probability predictive rules. We evaluate the performance of the main spatial disease cluster algorithms with these new metrics. Our analysis and experiments offer insights into when the usual metrics are not appropriate and also show that our proposal is very effective at eliminating the bias from the usual metrics.