MCoq: Mutation Analysis for Coq Verification Projects

Abstract

Software developed and verified using proof assistants, such as Coq, can provide trustworthiness beyond that of software developed using traditional programming languages and testing practices. However, guarantees from formal verification are only as good as the underlying definitions and specification properties. If properties are incomplete, flaws in definitions may not be captured during verification, which can lead to unexpected system behavior and failures. Mutation analysis is a general technique for evaluating specifications for adequacy and completeness, based on making small-scale changes to systems and observing the results. We demonstrate MCoq, the first mutation analysis tool for Coq projects. MCoq changes Coq definitions, with each change producing a modified project version, called a mutant, whose proofs are exhaustively checked. If checking succeeds, i.e., the mutant is live, this may indicate specification incompleteness. Since proof checking can take a long time, we optimized MCoq to perform incremental and parallel processing of mutants. By applying MCoq to popular Coq libraries, we found several instances of incomplete and missing specifications manifested as live mutants. We believe MCoq can be useful to proof engineers and researchers for analyzing software verification projects. The demo video for MCoq can be viewed at: Https://youtu.be/QhigpfQ7dNo.