Rollback mechanism of nested virtual machines for protocol fuzz testing

Abstract

Secure communications (HTTPS, SSH, etc) are important in the current Internet services. Implementations of secure protocols should be tested as exhaustively as possible. Repeated protocol fuzz testing from every reachable state is necessary and snapshot/rollback mechanism is required. Ordinary snapshot tools, however, only bring back a state of process or virtual machine (VM), and do not take care of packets on a wire. It means that they have no feature of distributed snapshot defined by Chandy-Lamport. Furthermore, secure protocols inherently depend upon a computing environment (e.g., random number) and make it difficult to repeat same testing. In order to solve these problems easily and generally, we propose a new protocol for controlling snapshot/rollback of VM, and an implementation which uses nested VMs and proxies. The internal VM of nested VM emulates whole hardware for exact repeat of protocol handling, and the external VM and proxies work for managing the state of internal VM and packets on a wire. In the current implementation internal VM is the instruction emulator QEMU and external VM is KVM which uses virtualization instructions. On a feasibility study, 4 TLS 1.2 servers (OpenSSL, GnuTLS, CyaSSL, and PolarSSL) were verified, and we found 2 bugs in CyaSSL and 1 bug in PolarSSL. Copyright 2014 ACM.