Evaluating obfuscation security: A quantitative approach

Abstract

State of the art obfuscation techniques rely on an unproven concept of security, therefore it is very hard to evaluate their protection quality. In previous work we introduced algorithmic information theory as a theoretical foundation for code obfuscation security.We propose Kolmogorov complexity, estimated by compression, as a software complexity metric to measure regularities in obfuscated programs. In this paper we provide a theoretical validation for its soundness as a software metric, so it can have as much credibility as other complexity metrics. Then, we conduct an empirical evaluation for 43 obfuscation techniques, which are applied to 10 Java byte code programs of SPECjvm2008 benchmark suite using three different decompilers as a threat model, aiming to provide experimental evidence that support the formal treatments.