FuzzTastic: A Fine-grained, Fuzzer-agnostic Coverage Analyzer

6Citations
Citations of this article
5Readers
Mendeley users who have this article in their library.

Abstract

Performing sound and fair fuzzer evaluations can be challenging, not only because of the randomness involved in fuzzing, but also due to the large number of fuzz tests generated. Existing evaluations use code coverage as a proxy measure for fuzzing effectiveness. Yet, instead of considering coverage of all generated fuzz inputs, they only consider the inputs stored in the fuzzer queue. However, as we show in this paper, this approach can lead to biased assessments due to path collisions. Therefore, we developed FuzzTastic, a fuzzer-agnostic coverage analyzer that allows practitioners and researchers to perform uniform fuzzer evaluations that are not affected by such collisions. In addition, its time-stamped coverage-probing approach enables frequency-based coverage analysis to identify barely tested source code and to visualize fuzzing progress over time and across code. To foster further studies in this field, we make FuzzTastic, together with a benchmark dataset worth ~12 CPU-years of fuzzing, publicly available; the demo video can be found at https://youtu.be/Lm-eBx0aePA.

References Powered by Scopus

An empirical study of the Reliability of UNIX Utilities

900Citations
N/AReaders
Get full text

Evaluating fuzz testing

489Citations
N/AReaders
Get full text

CollAFL: Path Sensitive Fuzzing

370Citations
N/AReaders
Get full text

Cited by Powered by Scopus

On the Reliability of Coverage-Based Fuzzer Benchmarking

53Citations
N/AReaders
Get full text

SoK: Prudent Evaluation Practices for Fuzzing

14Citations
N/AReaders
Get full text

Reachable Coverage: Estimating Saturation in Fuzzing

6Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Lipp, S., Elsner, D., Hutzelmann, T., Banescu, S., Pretschner, A., & Bohme, M. (2022). FuzzTastic: A Fine-grained, Fuzzer-agnostic Coverage Analyzer. In Proceedings - International Conference on Software Engineering (pp. 75–79). IEEE Computer Society. https://doi.org/10.1109/ICSE-Companion55297.2022.9793832

Readers' Seniority

Tooltip

Professor / Associate Prof. 1

50%

PhD / Post grad / Masters / Doc 1

50%

Readers' Discipline

Tooltip

Computer Science 1

100%

Save time finding and organizing research with Mendeley

Sign up for free