Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems

5Citations
Citations of this article
11Readers
Mendeley users who have this article in their library.

Abstract

Sample-and-rank is a key decoding strategy for modern generation-based dialogue systems. It helps achieve diverse and high-quality responses by selecting an answer from a small pool of generated candidates. The current state-of-the-art ranking methods mainly use an encoding paradigm called Cross-Encoder, which separately encodes each context-candidate pair and ranks the candidates according to their fitness scores. However, Cross-Encoder repeatedly encodes the same lengthy context for each candidate, resulting in high computational costs. Poly-Encoder addresses the above problems by reducing the interaction between context and candidates, but with a price of performance drop. In this work, we develop a new paradigm called Uni-Encoder, that keeps the full attention over each pair as in Cross-Encoder while only encoding the context once, as in Poly-Encoder. Uni-Encoder encodes all the candidates with the context in one forward pass. We use the same positional embedding for all candidates to ensure they are treated equally and design a new attention mechanism to avoid confusion. Our Uni-Encoder can simulate other ranking paradigms using different attention and response concatenation methods. Extensive experiments show that our proposed paradigm achieves new state-of-the-art results on four benchmark datasets with high computational efficiency. For instance, it improves R10@1 by 2.9% with an approximately 4× faster inference speed on the Ubuntu V2 dataset.

Cite

CITATION STYLE

APA

Song, C., He, H., Yu, H., Fang, P., Cui, L., & Lan, Z. (2023). Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (pp. 6231–6244). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.findings-acl.388

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free