DSPM-NLG: A Dual Supervised Pre-trained Model for Few-shot Natural Language Generation in Task-oriented Dialogue System

Abstract

In few-shot settings, fully conveying the semantic information of dialogue act is a crucial challenge for Natural Language Generation (NLG) in task-oriented dialogue systems. It is noteworthy that NLG and Spoken Language Understanding (SLU) form a natural dual problem pair. If the SLU module can successfully restore the generated response by the NLG module to the corresponding dialogue act, this would demonstrate that the response is effectively conveying the semantic information of the dialogue act. Based on this idea, a novel Dual Supervised Pre-trained Model for a few-shot Natural Language Generation (DSPM-NLG) is proposed to regularize the pre-training process. We adopt a joint model with a dual supervised framework to learn the dual correlation between NLG and SLU from a probabilistic perspective. In addition, a slot-masked strategy is designed to enable the model to focus more effectively on the key slot-value pairs. DSPM-NLG is continuously trained on publicly available and large-scale labeled data, allowing it to gain a thorough understanding of the duality between the two tasks and to enhance the pre-trained model's ability for semantic control and generalization. Experimental results illustrate that our proposed model demonstrates exceptional performance on the few-shot benchmark dataset, outperforming the previous state-of-the-art results.