Detecting metamorphic virus using Hidden Markov Model and genetic algorithm

Abstract

Metamorphic viruses dodges the classical signature-based detection system by modifying internal structure without compromising on the original functionality. To solve this problem, some machine learning technique, like Hidden Markov model (HMM) and Neural Network are can be used. HMM is a state machine where each state observes the input data with appropriate observation probability. HMM learns statistical properties of antivirus features rather than signatures and relies on such statistics to detect same family virus. Each HMM when trained with variants of same family viruses that are generated by same metamorphic engine so that HMM can detect similar viruses with high probability. But, in order to make the HMM detect viruses, there are three basic criteria that needs to be satisfied. Generally in most of the HMM based techniques, Baum-Welch method is used for solving one of the three problems, i.e, estimating the parameters of the corresponding HMM given an output sequence. In this paper, we have used the Genetic Algorithm to solve the problem. The selection of Genetic algorithm over the conventional Baum- Welch method lies in the non-linearity of the genetic algorithm. The Baum-Welch algorithm, being linear in nature, suffers from the local optima problem, which we have tried to overcome using our scheme. © 2012 Springer India Pvt. Ltd.