Prevent: An Unsupervised Approach to Predict Software Failures in Production

Abstract

This paper presents Prevent, a fully unsupervised approach to predict and localize failures in distributed enterprise applications. Software failures in production are unavoidable. Predicting failures and locating failing components online are the first steps to proactively manage faults in production. Many techniques predict failures from anomalous combinations of system metrics with supervised, weakly supervised, and semi-supervised learning models. Supervised approaches require large sets of labelled data not commonly available in large enterprise applications, and address failure types that can be either captured with predefined rules or observed while training supervised models. Prevent integrates the core ingredients of unsupervised approaches into a novel fully unsupervised approach to predict failures and localize failing resources. The results of experimenting with Prevent on a commercially-compliant distributed cloud system indicate that Prevent provides more stable, reliable and timely predictions than supervised learning approaches, without requiring the often impractical training with labeled data.