Robust reasoning over heterogeneous textual information for fact verification

Abstract

Automatic fact verification (FV) based on artificial intelligence is considered as a promising approach which can be used to identify misinformation distributed on the web. Even though previous FV using deep learning have made great achievements in single dataset (e.g., FEVER), the trained systems are unlikely to be capable of extracting evidence from heterogeneous web-sources and validating claims in accordance with evidence found on the Internet. Nevertheless, the heterogeneity covers abundant semantic information, which will help FV system identify misinformation in a more accurate way. The current work is the first attempt to make the combination of knowledge graph (KG) and graph neural network (GNN) to enhance the robustness of FV systems for heterogeneous information. As a result, it can be generalized to multi-domain datasets after training on a sufficient single one. To make information update and aggregate well on the collaborative graph, the present study proposes a double graph attention network (DGAT) framework which recursively propagates the embeddings from a node's neighbors to refine the node's embedding as well as applies an attention mechanism to classify the importance of the neighbors. We train and evaluate our system on FEVER, a single and benchmark dataset for FV, and then re-evaluate our system on UKP Snopes Corpus, a new richly annotated corpus for FV tasks on the basis of heterogeneous web sources. According to experimental results, although DGAT has no excellent advantages in a single dataset, it shows outstanding performance in more realistic and multi-domain datasets. Moreover, the current study also provides a feasible method for deep learning to have the ability to infer heterogeneous information robustly.