Learning to Consider Relevance and Redundancy Dynamically for Abstractive Multi-document Summarization

Abstract

As one of the most essential tasks for information aggregation, multi-document summarization is faced with information redundancy of source document clusters. Recent works have attempted to avoid redundancy while generating summaries. Most state-of-the-art multi-document summarization systems are either extractive or abstractive with an external extractive model. In this paper, we propose an end-to-end abstractive model based on Transformer to generate summaries, considering relevance and redundancy dynamically and jointly. Specifically, we employ sentence masks and design a sentence-level transformer layer for learning sentence representations in a hierarchical manner. Then we use a dynamic Max Marginal Relevance (MMR) model to discern summary-worthy sentences and modify the encoder-decoder attention. We also utilize the pointer mechanism, taking the mean attention of all transformer heads as the probability to copy words from the source text. Experimental results demonstrate that our proposed model outperforms several strong baselines. We also conduct ablation studies to verify the effectiveness of our key mechanisms.