GenCoG: A DSL-Based Approach to Generating Computation Graphs for TVM Testing

Abstract

TVM is a popular deep learning (DL) compiler. It is designed for compiling DL models, which are naturally computation graphs, and as well promoting the efficiency of DL computation. State-of-the-art methods, such as Muffin and NNSmith, allow developers to generate computation graphs for testing DL compilers. However, these techniques are inefficient - their generated computation graphs are either type-invalid or inexpressive, and hence not able to test the core functionalities of a DL compiler. To tackle this problem, we propose GenCoG, a DSL-based approach to generating computation graphs for TVM testing. GenCoG is composed of (1) GenCoGL, a domain-specific language for specifying type constraints of operators, and (2) an approach that concolically solves type constraints and incrementally generates computation graphs of high expressivity. We implement and evaluate GenCoG on TVM releases. Our results show that GenCoG is effective in generating valid and expressive computation graphs - all of the GenCoG-generated graphs pass type checking, a critical graph validation stage; letting the graphs' expressivity be measured by their vertex and edge diversities, GenCoG outperforms state-of-the-arts by achieving 1.65∼6.93× in vertex diversity and 1.06∼7.08× in edge diversity, respectively. Furthermore, GenCoG has detected 16 bugs in TVM v0.8 and v0.9, with 14 confirmed and 12 fixed.