Detecting Fake News Using BERT Word Embedding, Attention Mechanism, Partition and Overlapping Text Techniques

Abstract

More and more humans are using the World Wide Web through social networking applications and websites to easily transmit, publish, and interact with news because it is nearly open to everyone who wants to access it. To carry out evil intentions politically, economically, and ethically, the spread of misinformation leads individuals toward other undesirable agendas, and the other drawback is the diminishing confidence of community members in widely circulated news. This study finds an approach capable of diagnosing fake news depending on its textual content. It started with a Bidirectional Encoder Representations from Transformers (BERT) model to embed words. Texts of more than 512 words are partitioned into parts with a maximum length part of 512, and each part is embedded; an average embedding of parts is preserved as an embedding of the original text. The following step is selecting an idea that involves one convolutional layer sending its outputs to two parallel layers Convolutional layer and Bidirectional Long Short-Term Memory (BiLSTM) layer. The outputs of two separate parallel layers are sent into different attention layers, which combine the results of both layers before putting them into a fully connected layer that uses the sigmoid activation function.