SKALD: A scalable architecture for feature extraction, multi-user analysis, and real-time information sharing

Abstract

The inability of existing architectures to allow corporations to quickly process information at scale and share knowledge with peers makes it difficult for malware analysis researchers to present a clear pic- ture of criminal activity. Hence, analysis is limited in effectively and accurately identify the full scale of adversaries’ activities and develop effective mitigation strategies. In this paper, we present Skald: a novel architecture which guides the creation of analysis systems to support the research of malicious activities plaguing computer systems. Our design provides the scalability, flexibility, and robustness needed to process cur- rent and future volumes of data. We show that our prototype is able to process millions of samples in only few milliseconds per sample with zero critical errors. Additionally, Skald enables the development of new methodologies for information sharing, enabling analysis across collective knowledge. Consequently, defenders can perform accurate investigations and real-time discovery, while reducing mitigation time and infrastruc- ture cost.