Time-series aware precision and recall for anomaly detection considering variety of detection result and addressing ambiguous labeling

Abstract

We propose time-series aware precision and recall, which are appropriate for evaluating anomaly detection methods in time-series data. In time-series data, an anomaly corresponds to a series of instances. The conventional metrics, however, overlook this characteristic, so they suffer from a problem of giving a high score to the method that only detects a long anomaly. To overcome the problem, our metrics consider the variety of the detected anomalies to be more important through two scoring strategies, detection scoring (i.e., how many anomalies are detected) and portion scoring (i.e., how precisely each anomaly is detected). Moreover, our metrics concern ambiguous instances, which indicate the instances labeled as'normal' although they are affected by their precedent anomaly. Our metrics give smaller scores to those instances as they are likely to be anomalous. We demonstrate that our metrics are more suitable for time-series data compared to existing metrics by evaluations using a real-world dataset as well as several examples.