A class-level test selection approach toward full coverage for continuous integration

Abstract

Continuous Integration (CI) is an important practice in agile development. With the growth of integration system, running all tests to verify the quality of submitted code, is clearly uneconomical. This paper aims at selecting a proper test subset towards full coverage of all changed and affected code so as to reduce the cost of CI testing. We proposes FEST, a novel approach, which searches for the full dependencies of changed code at the class level and then selects test classes related to the changed and affected classes. We assess FEST from fault detection efficiency and cost effectiveness based on 18 open source projects with 261 continuous integration versions from Eclipse and Apache communities, and compare it with the state-of-the-art approach ClassSRTS (as baseline). Results show that FEST (1) can not only cover all faults detected by actual CI testing and baseline, but also find new faults in 25% and 18% versions respectively. (2) shows better or equal test scale benefits than actual CI testing (in 98% versions) and baseline (in 99% versions); and can compensate risk of omitting necessary tests for actual CI testing (in 62% versions) and baseline (in 73% versions).