The study on integer overflow vulnerability detection in binary executables based upon genetic algorithm

Abstract

The automatic identification of security vulnerabilities in the binary code is still a young but important research area for the security researchers. In recent years, the number of identified integer overflow vulnerabilities has been increasing rapidly. In this paper, we present a smart software vulnerability detection technology, which is used for the identification of integer overflow vulnerabilities in the binary executables. The proposed algorithm is combined with debugger module, static analysis module and genetic algorithm module. We use the fitness function to guide the generation of the tested data and use static analysis to provide the information that the genetic module needs. Theory analyses and experiment results indicate that the detection technology based upon genetic algorithm can identify the exceptions in the object program and is more efficient than the common Fuzzing technology. © 2011 Springer-Verlag Berlin Heidelberg.