Analysis of potential code vulnerabilities involving overlapping instructions

Abstract

This article proposes approaches supporting the analysis of code vulnerabilities based on overlapping machine instructions of variable length. For the purpose of focusing the search for potential malicious code it is suggested to apply first disassembling techniques allowing for a restriction of potentially exploitable memory space. Successively, testing based on heuristic optimization may be applied in order to evaluate dynamically the practicality of vulnerability exploitation.