Reducing Wrong Labels for Distantly Supervised Relation Extraction with Reinforcement Learning

Abstract

Relation extraction (RE) aims to mine semantic relations between entity pairs from plain texts, which plays an important role in various natural language processing (NLP) tasks. However, the existing methods in distant supervision (DS) are sensitive to bags and fail to handle sentence-level relation prediction. In particular, few methods focus on the sentence-level label denoising. In this paper, the sentence-level label denoising model based on reinforcement learning (RL) and the express-only-one assumption is proposed for distantly supervised RE. First, unlike removing the noisy sentences in previous studies, this paper designs Deep Q Network (DQN), a value-based RL algorithm, as a label denoiser to select the most reliable labels from the multiple relations that sentences are labeled. Second, the relation extractor applies the typical neural network model to predict relations between the data before and after the label denoiser cleans. The rewards in label denoiser are measured by the differences of prediction scores. Finally, the two modules between label denoiser and relation extractor are trained jointly to obtain correct labels and improve the extraction performance at the sentence level. The experimental results show that the proposed denoiser can deal with the noise labels of data effectively and the proposed model outperforms previous state-of-the-art baselines on both the Riedel dataset and human-annotated dataset.