Packer identification using hidden markov model

Abstract

Most of modern malware are packed by packers to evade the anti-virus software. Basically, packers will apply various obfuscating techniques to hide their true behaviors from static analysis methods. Thus, how to deal with packed malware has always been a tough problem so far. This paper proposes a novel approach for packer detection using a combination of BE-PUM tool and Hidden Markov Model. First, BE-PUM tool is applied to detect the sequence of possible obfuscation techniques embedded in the analyzed binary program. Then, Hidden Markov Model is used to effectively identify the possibility of packer existence from the generated sequences. As Hidden Markov is very effective for pattern recognition, our proposed technique can accurately identify the packers deployed in binaries files. We have performed experiments on more than 2000 real-world malwares taken from VirusShare. The result is very promising.