Automated software vulnerability analysis

Abstract

Despite decades of research, software continues to have vulnerabilities. Successful exploitations of these vulnerabilities by attackers cost millions of dollars to businesses and individuals. Unfortunately, most effective defensive measures, such as patching and intrusion prevention systems, require an intimate knowledge of the vulnerabilities. Many systems for detecting attacks have been proposed. However, the analysis of the exploited vulnerabilities is left to security experts and programmers. Both the human effort involved and the slow analysis process are unfavorable for timely defensive measure to be deployed. The problem is exacerbated by zero-day attacks. This chapter presents two recent research efforts, named MemSherlock and CBones, for automatically aiding experts in identifying and analyzing unknown vulnerabilities. Both methods rely on monitoring user applications during their runtime and checking for inconsistencies in their memory or memory access patterns. MemSherlock is a post-mortem analysis tool that monitors an application's memory operations to determine malicious ones, indicative of an ongoing attack. It produces valuable information regarding the vulnerability and the attack vector. CBones takes snapshots of the memory and looks for inconsistencies by identifying invariants for an application's memory and verifying them at runtime. Experimental evaluation shows that both methods are capable of providing critical information about vulnerabilities and attack vectors. © Springer Science+Business Media, LLC 2010.