PARDIS: Priority aware test case reduction

Abstract

Test cases play an important role in testing and debugging software. Smaller tests are easier to understand and use for these tasks. Given a test that demonstrates a bug, test case reduction finds a smaller variant of the test case that exhibits the same bug. Classically, one of the challenges for test case reduction is that the process is slow, often taking hours. For hierarchically structured inputs like source code, the state of the art is Perses, a recent grammar aware and queue driven approach for test case reduction. Perses traverses nodes in the abstract syntax tree (AST) of a program (test case) based on a priority order and tries to reduce them while preserving syntactic validity. In this paper, we show that Perses’ reduction strategy suffers from priority inversion, where significant time may be spent trying to perform reduction operations on lower priority portions of the AST. We show that this adversely affects the reduction speed. We propose, a technique for priority aware test case reduction that avoids priority inversion. We implemented and evaluated it on the same set of benchmarks used in the Perses evaluation. Our results indicate that compared to Perses, is able to reduce test cases 1.3x to 7.8x faster and with 46% to 80% fewer queries.