An effective deep learning approach for extractive text summarization

Abstract

Nowadays, most research on extractive text summarization uses deep learning approaches as they provide better performances than the others. However, a difficulty in these approaches is the shortage of a large dataset for training summarization systems. To deal with this problem, we take advantage of contextualized word embeddings from pre-trained BERT models to produce sentence embedding vectors. These vectors are then used as the input of a Multi-Layer Perceptron classifier for sentence selection. The outputs of the Multi-Layer Perceptron classifier are processed by a Maximal Marginal Relevance algorithm to remove redundant sentences. Finally, the selected sentences are rearranged using information about sentence position in the original document to create a summary. Our proposed system is evaluated by using both English and Vietnamese datasets. Experimental results show that our system achieves promising results comparing to existing researches in this field.