What makes a code review trustworthy?

Abstract

Code review is an important step during the process of certifying safety-critical software because only code that passes review can be implemented. Reviews are performed by review boards composed of highly skilled and experienced computer scientists, engineers and analysts who generally rely upon a checklist of properties ranging from high-level requirements to minute language details. While many checklists and coding standards exist, the actual decision of which properties are most important is generally based on the experience of the person in charge. This paper addresses the questions: How can code review ensure certification of trustworthy code? and Is code review trustworthy? We surveyed technical leaders at NASA and the Aerospace industry to find out which properties are most important during the code review. To make analyze easier, the most common properties have been classified along different "views", ranging from a standards-oriented view (defined as the properties needed to satisfy a specific standard) to a tool-oriented view. In this paper, we present this classification together with a summary of findings and feed-back from the survey. We also discuss how a more uniform view on properties of code review and tool capabilities can result in increased trust for safety-critical software.