Evaluating the Strategies of Statement Selection in Automated Program Repair

Abstract

Automated program repair has drawn significant attention in recent years for its potential to alleviate the heavy burden of debugging activities. Fault localization, which generally provides a rank list of suspicious statements, is necessary for automated repair tools to identify the fault. With such rank list, existing repair tools have two statement selecting strategies for statement modification: suspiciousness-first algorithm (SFA) based on the suspiciousness of statements and rank-first algorithm (RFA) relying on the rank of statements. However, despite the extensive application of SFA and RFA in repair tools and different selecting methods between both strategies, little is known about the performance of the two strategies in automated program repair. In this paper we conduct an empirical study to compare the effectiveness of SFA and RFA in automated program repair. We implement SFA and RFA as well as 6 popular spectrum-based fault localization techniques into 4 automated program repair tools, and then use these tools to perform repair experiments on 44 real-world bugs from Defects4J to compare SFA against RFA in automated program repair. The results suggest that: (1) RFA performs statistically significantly better than SFA in 64.76% cases according to the number of candidate patches generated before a valid patch is found (NCP), (2) SFA achieves better performance in parallel repair and patch diversity, (3) The performance of SFA can be improved by increasing the suspiciousness accuracy of fault localization techniques. These observations provide directions for future research on the usage of statement selecting strategies and a new avenue for improving the effectiveness of fault localization techniques in automated program repair.