Protocol reverse engineering: Challenges and obfuscation

Abstract

Reverse engineering of communication protocols is aimed at providing methods and tools allowing to infer a model of these protocols. It is very relevant for many application domains, such as interoperability or security audits. Recently, several tools have been developed in order to automate, entirely or partially, the protocol inference process. These tools rely on several techniques, that are usually tuned and adapted according to the final goal of the reverse engineering task. The aim of this paper is (1) to present an overview of the main challenges related to reverse engineering, and (2) to introduce the use of obfuscation techniques to make the reverse engineering process more complex and difficult in particular to malicious users.