Denoising Distant Supervision for Relation Extraction with Entropy Weight Method

Abstract

Distant supervision for relation extraction has been widely used to construct training set by aligning the triples of the knowledge base, which is an efficient method to reduce human efforts. However, this method inevitably suffers from wrong labeling problems leading too much noise that will severely hurt the performance of relation extraction. To tackle this problem, in this paper, we propose a denoising model based on Entropy Weight Method (EWM) to filter the noise and select most relevant sentences. First, in a pretraining stage, we develop a sentence-level relation aware attention mechanism to distinguish several most relevant sentence, increasing the attention weights for those critical sentences. Second, we filter the noisy sentences by calculating the entropy weight using the above attention matrix, and then we employ intra-bag and inter-bag attentions to aggregate these selected sentence representations. Experiments on the NYT dataset show that our method can significantly reduce the noisy instance and achieve the state-of-the-art model performance.