Semantically non-preserving transformations for antivirus evaluation

Abstract

We relax the notion of malware obfuscation to include semantically non-preserving transformations. Unlike traditional obfuscation techniques, these transformation may not preserve original code behaviour. Using web-based malware we focus on transformations which modify abstract syntax trees. While such transformations yield syntactically valid programs, they may yield dysfunctional samples, so that it is not clear that this is a practical approach to producing detection-evading malware. However, by implementing an automated system that efficiently filters dysfunctional samples on a virtual cloud architecture, we show that such transformations are in fact practical. Using two simple transformations, we evaluated four antivirus products and were able to create many samples that evade detection, demonstrating that semantic-preserving obfuscation is not the only effective way to mutate malware.