Building and evaluating an annotated corpus for automated recognition of chat‐based social engineering attacks

Abstract

Chat‐based Social Engineering (CSE) is widely recognized as a key factor to successful cyber‐attacks, especially in small and medium‐sized enterprise (SME) environments. Despite the interest in preventing CSE attacks, few studies have considered the specific features of the language used by the attackers. This work contributes to the area of early‐stage automated CSE attack recognition by proposing an approach for building and annotating a specific‐purpose corpus and presenting its application in the CSE domain. The resulting CSE corpus is then evaluated by training a bi‐directional long short‐term memory (bi‐LSTM) neural network for the purpose of named entity recognition (NER). The results of this study emphasize the importance of adding a plethora of metadata to a dataset to provide critical in‐context features and produce a corpus that broadens our understanding of the tactics used by social engineers. The outcomes can be applied to dedicated cyber‐defence mechanisms utilized to protect SME employees using Electronic Medium Communication (EMC) software.