Regression test case prioritisation for Guava

Abstract

We present a three objective formulation of regression test prioritisation. Our formulation involves the well-known, and widely-used objectives of Average Percentage of Statement Coverage (APSC) and Effective Execution Time (EET). However, we additionally include the Average Percentage of Change Coverage (APCC), which has not previously been used in search-based regression test optimisation. We apply our approach to prioritise the base and the collection package of the Guava project, which contains over 26,815 test cases. Our results demonstrate the value of search-based test case prioritisation: the sequences we find require only 0.2% of the 26,815 test cases and only 0.45% of their effective execution time. However, we find solutions that achieve more than 99.9% of both regression testing objectives; covering both changed code and existing code. We also investigate the tension between these two objectives for Guava.