Increasing the Accuracy of Malware Detection in PE Files Using Advanced Machine-Learning Techniques

Abstract

This work presents the design and implementation of a new malware detection system for Portable Executable (PE) files in the Windows operating system using machine learning methods. The system uses various techniques for extracting attributes from binary files, including the n-gram method, branch analysis of instructions, and frequency analysis of instructions, which are combined with information about imported functions. To improve the classification accuracy and optimize the computational effort, neural networks with different architectures were trained and optimized. The implemented API (Application Programming Interface) interface allows easy integration of the system into existing security solutions and offers flexibility in the choice of classification models. Experimental results are compared with commercial detection systems, while the detection accuracy and false positive and false negative rates are analyzed. The limitations of the proposed approach, possibilities for its improvement, and perspectives for further use of machine learning in the field of malware detection are discussed in the conclusion of the work.