Semantic set analysis for malware detection

Abstract

Nowadays, malware is growing rapidly through the last few years and becomes more and more sophisticated as well as dangerous. A striking malware is obfuscation malware that is very difficult to detect. This kind of malware can create new variants that are similar to original malware feature but different about code. In order to deal with such types of malware, many approaches have been proposed, however, some of these approaches are ineffective due to their limited detection range, huge overheads or manual stages. Malware detection based on signature, for example, cannot overcome the obfuscation techniques of malware. Likewise, the behavior-based methods have the natural problems of a monitoring system such as recovery costs and longlasting detection time. In this paper, we propose a new method (semantic set method) to detect metamorphic malware effectively by using semantic set (a set of changed values of registers or variables allocated in memory when a program is executed). For more details, this semantic set is analyzed by n-gram separator and Naïve Bayes classifier to increase detection accuracy and reduce detection time. This system has been already experimented on different datasets and got the accuracy up to 98% and detection rate almost 100%.