Syntax and behavior semantics analysis of network protocol of malware

Abstract

Network protocol reverse analysis is an important aspect of malware analysis. There are many different network protocols and every protocol contains different types of fields that result in various malware behaviors. Without the protocol syntax and filed semantics, analyzers cannot understand how malware interacts with the outside network. This paper presents a syntax and a behavior semantics analysis method of the network protocol. By monitoring the way malware parse the network data and by using different fields in a virtual execution environment, this method can identify protocol fields, extract protocol syntax and correlate each syntax with malware behaviors, accordingly. This paper designs and implements the prototype Prama (protocol reverse analyzer for malware analysis). Experimental results show that this method can correctly infer protocol syntax and tag fields with meaningful malware behaviors. © Copyright 2011, Institute of Software, the Chinese Academy of Sciences.