Imbalanced Stance Detection by Combining Neural and External Features

Abstract

Stance detection is the task of determining the perspective “or stance” of pairs of text. Classifying the stance (e.g. agree, disagree, discuss or unrelated) expressed in news articles with respect to a certain claim is an important step in detecting fake news. Many neural and traditional models predict well on unrelated and discuss classes while they poorly perform on other minority represented classes in the Fake News Challenge-1 (FNC-1) dataset. We present a simple neural model that combines similarity and statistical features through a MLP network for news-stance detection. Aiding augmented training instances to overcome the data imbalance problem and adding batch-normalization and gaussian-noise layers enable the model to prevent overfitting and improve class-wise and overall accuracy. We also conduct additional experiments with a light-GBM and MLP network using the same features and text augmentation to show their effectiveness. In addition, we evaluate the proposed model on the Argument Reasoning Comprehension (ARC) dataset to assess the generalizability of the model. The experimental results of our models outperform the current state-of-the-art.