Secure and efficient in-process monitor (and Library) protection with Intel MPK

Abstract

The process reference monitor is a common technique to enforce security policies for application execution. Reference monitors can be used to enforce access control, check program integrity, detect attacks and even transform program states. Deciding where the monitor resides involves a trade-off between strong monitor isolation and low switching overheads. Running the monitor in the same address space as the protected/traced application (in-process monitors) allows for low overhead but raises isolation concerns. Thus, existing work place monitors in a separate address space, which leads to expensive monitor invocation cost. We present MonGuard, a system in which a high-performance in-process monitor is efficiently isolated from the rest of the application. To that aim, we leverage the Intel Memory Protection Key (MPK) technology to enforce execute-only memory, combined with code randomization to protect and hide the monitor. MonGuard inserts instrumentation around sensitive instructions to further prevent possible code reuse attacks. We built a prototype of MonGuard as a loader extension and implemented a multi-variant execution (MVX) monitor. The evaluation shows MonGuard enhances the monitor protection with nearly zero performance overhead.