This malware looks familiar: Laymen identify malware run-time similarity with chernoff faces and stick figures

Abstract

Classifying unknown malicious binaries into malware families provides valuable information to security professionals. The task of recognizing a malicious binary (i.e., atributing it to a previously observed atack patern) is widely considered a difficult task requiring extensive domain expertise. In this work, we offer a new approach which focuses on transforming the the recognition problem domain from system traces to Chernoff faces, thereby engaging the facial recognition aptitude of laymen. To do so we (i) curated a expert tagged dataset of malware variants, (ii) instrumented behavior trace monitors for each variant, (iii) constructed a simple, graph based feature set from the runtime behavior, and (iv) visualized low-dimensional representations of these system call graphs with stick figures and Chernoff faces. We then selected the three families with the largest variation and asked non-experts on Amazon Mechanical Turk to classify binaries between these three families using the generated visual representations, a task that would otherwise be delegated to experts. We found that non-experts completed the task with between 63% and 86% accuracy, and when aggregated, these non-expert labels successfully trained a classifier to a similar level of performance as the ground truth labels. Although simple, the experiments conducted provide a novel evaluation of the inherent difficulty of malware recognition tasks. Additionally new operational possibilities for effective human in the loop malware recognition are indicated and discussed as future work within the research prospectus.