How Good is Static Analysis at Finding Concurrency Bugs?
Devin Kester, Martin Mwebesa, Jeremy S. Bradbury
Software Quality Research Group
Faculty of Science (Computer Science)
University of Ontario Institute of Technology
Oshawa, Ontario, Canada
devin.kester@mycampus.uoit.ca, fmartin.mwebesa, jeremy.bradburyg@uoit.ca
Abstract—Detecting bugs in concurrent software is challeng-
ing due to the many different thread interleavings. Dynamic
analysis and testing solutions to bug detection are often costly
as they need to provide coverage of the interleaving space
in addition to traditional black box or white box coverage.
An alternative to dynamic analysis detection of concurrency
bugs is the use of static analysis. This paper examines the use
of three static analysis tools (FindBugs, JLint and Chord) in
order to assess each tool’s ability to find concurrency bugs and
to identify the percentage of spurious results produced. The
empirical data presented is based on an experiment involving
12 concurrent Java programs.
Keywords-static analysis, concurrency, data race, deadlock,
empirical software engineering.
I. INTRODUCTION
The widespread adoption of multicore processors has lead
to an increase in demand for concurrent software that can
exploit the performance gains possible from utilizing more
than just a single core. For example, the Java programming
language, which has primarily been used to write sequential
programs is now often used to write multithreaded programs
using synchronization, locks, semaphores, barriers and ex-
changers. All of these language features aim to enhance the
sharing of data between threads in a Java program.
Although using concurrency can be beneficial with respect
to performance it can also be extremely difficult to do
correctly. The difficulty with concurrent programming has
been discussed since before multicore technology became
mainstream. The importance of this problem is reflected in
the following quote “Industry needs help from the research
community to succeed in its recent dramatic shift to parallel
computing. Failure could jeopardize both the IT industry
and the portions of the economy that depend on rapidly
improving information technology” [1]. One of the major
issues with concurrent programming is that it can lead to
kinds of bugs that are not possible in sequential programs.
For example, a concurrent program that shares data across
multiple threads can contain a bug that leads to a data
race or deadlock. In a language like Java, the interleaving
space of a concurrent program consists of all possible thread
schedules [2]. Detecting concurrency bugs like data races
or deadlocks is a challenging software quality assurance
problem because a particular bug may only exhibit itself
in one or a few interleavings of a program - such bugs are
often referred to as heisenbugs [3].
Although concurrency bugs are difficult to detect there are
tools that can significantly enhance the ability to find possi-
bly obscure concurrency bugs. One category of these tools
are static analysis tools like FindBugs [4], [5], JLint [6],
[7], RacerX [8], RELAY [9] and Chord [10], [11], [12].
Static analysis tools are typically considered to be fast since
execution of the program is not required. However, the use of
these tools has a potential for spurious analysis results which
can be time consuming to identify. Despite this shortcoming,
the effectiveness of static analysis tools in finding concur-
rency bugs can be clearly witnessed in the case of Coverity,
a very popular static analysis tool used in industry [13].
Another category of tools are dynamic analysis tools which
include testing and model checking tools like ConTest [14],
[15], CalFuzzer [16], CHESS [17] and Java PathFinder [18],
[19]. The dynamic analysis tools all involve executing the
program and in general are immune from spurious results.
Most dynamic concurrency bug detection tools include a
mechanism for executing different thread interleavings in
order to provide increased confidence in the quality of the
source code.
An important factor to consider when discussing static and
dynamic analysis tools is the increasing size and complexity
of programs under analysis. The size of concurrent software
offers a major challenge to the effectiveness of both static
and dynamic tools for detecting concurrency bugs [20]. The
successfully use of tools like Coverity and FindBugs in
industry provides evidence that static analysis tools can scale
to larger industrial software systems.
The goal of our research is to focus on the static analysis
class of bug detection tools. In particular, we are interesting
in answering the following questions:
 How effective are existing static analysis tools at de-
tecting concurrency bugs?
 What is the rate of false positives (spurious results) in
existing static analysis tools?
In order to answer the above questions we have conducted
an experiment which compares three existing tools that are
all capable of detecting concurrency problems in Java source