Symmetric-key cryptographic routine detection in anti-reverse engineered binaries using hardware tracing

Abstract

Software uses cryptography to provide confidentiality in communication and to provide authentication. Additionally, cryptographic algorithms can be used to protect software against cracking core algorithms in software implementation. Recently, malware and ransomware have begun to use encryption to protect their codes from analysis. As for the detection of cryptographic algorithms, previous works have had demerits in analyzing anti-reverse engineered binaries that can detect differences in analysis environments and normal execution. Here, we present a new symmetric-key cryptographic routine detection scheme using hardware tracing. In our experiments, patterns were successfully generated and detected for nine symmetric-key cryptographic algorithms. Additionally, the experimental results show that the false positive rate of our scheme is extremely low and the prototype implementation successfully bypasses anti-reversing techniques. Our work can be used to detect symmetric-key cryptographic routines in malware/ransomware with anti-reversing techniques.