Investigating the effectiveness of peer code review in distributed software development based on objective and subjective data

Abstract

Code review is a potential means of improving software quality. To be effective, it depends on different factors, and many have been investigated in the literature to identify the scenarios in which it adds quality to the final code. However, factors associated with distributed software development, which is becoming increasingly common, have been little explored. Geographic distance can impose additional challenges to the reviewing process. We thus in this paper present the results of a mixed-method study of the effectiveness of code review in distributed software development. We investigate factors that can potentially influence the outcomes of peer code review. The study involved an analysis of objective data collected from a software project involving 201 members and a survey with 50 practitioners with experience in code review. Our analysis of objective data led to the conclusion that a high number of changed lines of code tends to increase the review duration with a reduced number of messages, while the number of involved teams, locations, and participant reviewers generally improve reviewer contributions, but with a severe penalty to the duration. These results are consistent with those obtained in the survey regarding the influence of factors over duration and participation. However, participants’ opinion about the impact on contributions diverges from results obtained from historical data, mainly with respect to distribution.