Modeling Obfuscation Stealth Through Code Complexity

Abstract

Code obfuscation is often utilized by authors of malware to protect it from detection or to hide its maliciousness from code analysis. Obfuscation stealth describes how difficult it is to determine which protection technique has been applied to a program and which parts of the code have been protected. In previous literature, most of the presented obfuscation identification methods analyze the program code itself (for example, the frequency and distribution of opcodes). However, simple countermeasures such as instruction substitution can have a negative impact on the identification rate. In this paper, we present a novel approach for an accurate obfuscation identification model based on a combination of multiple code complexity metrics. An evaluation with 4124 samples protected with 11 different obfuscations, combinations of obfuscations, and various compiler configurations demonstrates an overall classification accuracy of 86.5%.