Reproducible network benchmarks with coNCePTuaL

Abstract

A cornerstone of scientific progress is the ability to reproduce experimental results. However, in the context of network benchmarking, system complexity impedes a researcher's attempts to record all of the information needed to exactly reconstruct a network-benchmarking experiment. Without this information, results may be misinterpreted and are unlikely to be reproducible. This paper presents a tool called CONCEPTUAL which simplifies most aspects of recording and presenting network performance data. CONCEPTUAL includes two core components: (1) a compiler for a highlevel, domain-specific programming language that makes it possible to specify arbitrary communication patterns tersely but precisely and (2) a complementary run-time library that obviates the need for writing (and debugging!) all of the mundane but necessary routines needed for benchmarking, such as those that calibrate timers, compute statistics, or output log files. The result is that CONCEPTUAL makes it easy to present network-performance data in a form that promotes reproducibility. © Springer-Verlag 2004.