Multi-task Learning for Paraphrase Generation With Keyword and Part-of-Speech Reconstruction

Abstract

Paraphrase generation using deep learning has been a research hotspot of natural language processing in the past few years. While pre vious studies tackle the problem from differ ent aspects, the essence of paraphrase gen eration is to retain the key semantics of the source sentence and rewrite the rest of the con tent. Inspired by this observation, we pro pose a novel two-stage model, PGKPR, for paraphrase generation with keyword and part-of-speech reconstruction. The rationale is to capture simultaneously the possible keywords of a source sentence and the relations between them to facilitate the rewriting. In the first stage, we identify the possible keywords using a pre diction attribution technique, where the words obtaining higher attribution scores are more likely to be the keywords. In the second stage, we train a transformer-based model via multi task learning for paraphrase generation. The novel learning task is the reconstruction of the keywords and part-of-speech tags, respectively, from a perturbed sequence of the source sen tence. The learned encodings are then decoded to generate the paraphrase. We conduct the experiments on two commonly-used datasets, and demonstrate the superior performance of PGKPR over comparative models on multiple evaluation metrics.