Defeating anti-debugging techniques for Malware analysis using a debugger

Abstract

Cyberattacks such as spear phishing and malspam pretending to be companies, institutes, and government officials are increasing and evolving. Malware has a variety of purposes, such as collecting personal information and illegal access to the system. New types of malware are increasing every day, and many malware programs spread all over the Internet, causing severe problems. To analyze such malware effectively, analysts first need to understand the inner structure of the malware. They can try to analyze malware manually and automatically. However, attackers who create malware use many different kinds of techniques, such as anti-reverse engineering, to hinder and delay analysis. They also extend malware life through a combination of different techniques, such as social engineering and anti-debugging. These techniques make the malware more sophisticated; thus, it is hard for an analyst to detect the malware. Anti-debugging, one way to protect malware, is a deadly poison to malware analysts because it makes the analysis more difficult by detecting a debugger or debugging environments. Therefore, this paper describes malware’s anti-debugging techniques and how to defeat them through anti-anti-debugging mechanisms. It applies its findings to analyze a sample program, packed files, and actual malware with anti-debugging modules and performs various experiments to verify the proposed techniques. After the experiments, it confirms whether its countermeasure is useful for malware analysis.