Eye movements in code review

Abstract

In order to ensure sufficient quality, software engineers conduct code reviews to read over one another's code looking for errors that should be fixed before committing to their source code repositories. Many kinds of errors are spotted, from simple spelling mistakes and syntax errors, to architectural flaws that may span several files. However, we know little about how software developers read code when looking for defects. What kinds of code trigger engineers to check more deeply into suspected defects? How long do they take to verify whether a defect is really there? We conducted a study of 35 software engineers performing 40 code reviews while capturing their gaze with an eye tracker. We classified each code defect the developers found and captured the patterns of eye gazes used to deliberate about each one. We report how long it took to confirm defect suspicions for each type of defect and the fraction of time spent skimming the code vs. carefully reading it. This work provides a starting point for automating code reviews that could help engineers spend more time focusing on the difficult task of defect confirmation rather than the tedious task of defect discovery.