System call API obfuscation

Abstract

We claim that attacks can evade the comprehension of security tools that rely on knowledge of standard system call interfaces to reason about process execution behavior. Our attack, called Illusion, will invoke privileged operations in a Windows or Linux kernel at the request of user-level processes without requiring those processes to call the actual system calls corresponding to the operations. The Illusion interface will hide system operations from user-, kernel-, and hypervisor-level monitors mediating the conventional system-call interface. Illusion will alter neither static kernel code nor read-only dispatch tables, remaining elusive from tools protecting kernel memory. © 2008 Springer-Verlag Berlin Heidelberg.