A practical method to confine sensitive API invocations on commodity hardware

Abstract

Control-flow hijacking attacks are a very dangerous threat to software security in that they can hijack the programs execution to execute malicious code. There have been many solutions proposed for countering these attacks, but majority of them suffer from the following limitations: (1) Some methods could be bypassed by advanced code reuse attacks; (2) Some methods will incur considerable performance cost; (3) Some methods need to modify the target program. To address these problems, we present APIdefender, a kernel-based solution to defeat control-flow attacks. Our method is compatible with the existing software and hardware. The basic idea of our approach is to confine the sensitive API invocations by comparing the invocation context with the baseline information that is obtained by offline analysis. To perform the run-time enforcement for the API invocations, we leverage some commodity hardware features. The experiments show that APIdefender can detect malicious API invocations effectively with a little performance overhead.