Bad and good news about using software assurance tools

Abstract

Software assurance tools – tools that scan the source or binary code of a program to find weaknesses – are the first line of defense in assessing the security of a software project. Even though there are a plethora of such tools available, with multiple tools for almost every programming language, adoption of these tools is spotty at best. And even though different tools have distinct abilities to find different kinds of weaknesses, the use of multiple tools is even less common. And when the tools are used (or attempted to be used), they are often used in ways that reduce their effectiveness. We present a step-by-step discussion of how to use a software assurance tool, describing the challenges that can occur in this process. We also present quantitative evidence about the effects that can occur when assurance tools are applied in a simplistic or naive way. We base this presentation on our direct experiences with using a wide variety of assurance tools. We then present the US Department of Homeland Security funded Software Assurance Marketplace (SWAMP), an open facility where users can upload their software to have it automatically and continually assessed by a variety of tools. The goal of the SWAMP is to simplify the task of the programmer in using assurance tools, thereby removing many of the obstacles to their adoption. Copyright © 2016 The Authors. Software: Practice and Experience Published by John Wiley & Sons, Ltd.