Towards Detecting Trigger-Based Behavior in Binaries: Uncovering the Correct Environment

Abstract

In this paper, we present our first results towards detecting trigger-based behavior in binary programs. A program exhibits trigger-based behavior if it contains undocumented, often malicious functionality that is executed only under specific circumstances. In order to determine the inputs and environment required to trigger such behavior, we use directed symbolic execution and present techniques to overcome some of its practical limitations. Specifically, we propose techniques to overcome the environment problem and the path selection problem. We implemented our techniques and evaluated their performance on a real malware sample that launches denial-of-service attacks upon receiving specific remote commands. Thanks to our techniques, our implementation was able to determine those specific commands and all other requirements needed to trigger the malicious behavior in reasonable time.