CASTSAN: Efficient detection of polymorphic C++ object type confusions with LLVM

Abstract

C++ object type confusion vulnerabilities as the result of illegal object casting have been threatening systems’ security for decades. While there exist several solutions to address this type of vulnerability, none of them are sufficiently practical for adoption in production scenarios. Most competitive and recent solutions require object type tracking for checking polymorphic object casts, and all have prohibitively high runtime overhead. The main source of overhead is the need to track the object type during runtime for both polymorphic and non-polymorphic object casts. In this paper, we present CastSan, a C++ object type confusion detection tool for polymorphic objects only, which scales efficiently to large and complex code bases as well as to many concurrent threads. To considerably reduce the object type cast checking overhead, we employ a new technique based on constructing the whole virtual table hierarchy during program compile time. Since CastSan does not rely on keeping track of the object type during runtime, the overhead is drastically reduced. Our evaluation results show that complex applications run insignificantly slower when our technique is deployed, thus making CastSan a real-world usage candidate. Finally, we envisage that based on our object type confusion detection technique, which relies on ordered virtual tables (vtables), even non-polymorphic object casts could be precisely handled by constructing auxiliary non-polymorphic function table hierarchies for static classes as well.