Denial-of-service attacks on host-based generic unpackers

Abstract

With the advance of packing techniques, a few generic and automatic unpackers have been proposed. These unpackers are designed to automatically unpack packed binaries without specific knowledge of the packing techniques used. In this paper, we present an automatic packer with which packed malware forges spurious unpacking behaviors that lead to a denial-of-service attack on host-based generic unpackers. We present the design, implementation, and evaluation of the proposed packer and malware produced using the proposed packer, and show the success of denial-of-service attacks on host-based generic unpackers. © 2009 Springer-Verlag.