A Preliminary Study on Personalized Spam E-mail Filtering Using Bidirectional Encoder Representations from Transformers (BERT) and TensorFlow 2.0

Abstract

Email security has been a major concern in for a long time. One important aspect of e-mail security is effective and efficient detection of spam e-mails, which is an added overhead in the proper functioning of modern email communication systems. In this paper, a method based the Bidirectional Encoders Representations from Transformers (BERT) is proposed that stems from deep learning (DL). Similar to Word Embedding, BERT is a technique for text representation and a combination of various DL methods such as bidirectional encoder LSTM and Transformers. The pre-training phase takes significant computational effort, and to save computational time, we used a pre-trained BERT model. A preliminary analysis of the proposed algorithm is carried out using a standard test suite of Enron dataset. Initial results indicate that the proposed algorithm has a potential of generating promising results.