Extractive summarization by rouge score regression based on bert

Abstract

In this paper, we present BESRES, a BERT-based regression model for extractive document summarization. We mainly explore two techniques for employing the BERT model for extractive document summarization: (1) Token Score Prediction: we propose to decompose the sentence score prediction into token-level score prediction, and (2) Soft Label: we propose a new attempt of using uses Rouge scores as an index for scoring the importance of a sentence. From experiment results, our model incorporating the two proposed techniques advances the state-of-the-art result on CNN Dailymail dataset from 42.99 to 43.42 in terms of Rouge-1 score.