ConVEx: Data-Efficient and Few-Shot Slot Labeling

24Citations
Citations of this article
89Readers
Mendeley users who have this article in their library.

Abstract

We propose ConVEx (Conversational Value Extractor), an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks. Instead of relying on more general pretraining objectives from prior work (e.g., language modeling, response selection), ConVEx’s pretraining objective, a novel pairwise cloze task using Reddit data, is well aligned with its intended usage on sequence labeling tasks. This enables learning domain-specific slot labelers by simply fine-tuning decoding layers of the pretrained general-purpose sequence labeling model, while the majority of the pretrained model’s parameters are kept frozen. We report state-of-the-art performance of ConVEx across a range of diverse domains and data sets for dialog slot-labeling, with the largest gains in the most challenging, few-shot setups. We believe that ConVEx’s reduced pretraining times (i.e., only 18 hours on 12 GPUs) and cost, along with its efficient fine-tuning and strong performance, promise wider portability and scalability for data-efficient sequence-labeling tasks in general.

References Powered by Scopus

Universal sentence encoder for English

763Citations
N/AReaders
Get full text

An introduction to conditional random fields

758Citations
N/AReaders
Get full text

The second dialog state tracking challenge

437Citations
N/AReaders
Get full text

Cited by Powered by Scopus

Controllable Semantic Parsing via Retrieval Augmentation

30Citations
N/AReaders
Get full text

Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

24Citations
N/AReaders
Get full text

Inverse is Better! Fast and Accurate Prompt for Few-shot Slot Tagging

20Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Henderson, M., & Vulić, I. (2021). ConVEx: Data-Efficient and Few-Shot Slot Labeling. In NAACL-HLT 2021 - 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference (pp. 3375–3389). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2021.naacl-main.264

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 28

65%

Researcher 10

23%

Professor / Associate Prof. 3

7%

Lecturer / Post doc 2

5%

Readers' Discipline

Tooltip

Computer Science 33

73%

Linguistics 6

13%

Social Sciences 4

9%

Engineering 2

4%

Save time finding and organizing research with Mendeley

Sign up for free