Automatic prevention of accidents in production

Abstract

We present a framework for automatically testing functional correctness of back-end servers. We created a pre-production environment where traffic between servers can be reconfigured dynamically. Production requests are sampled and replayed in our framework so that we can cover many corner cases of the system without having the developer manually write test cases. We also describe how to handle mutate requests and support checking the validity of back-end rewrites.