Know what you don't know: Modeling a pragmatic speaker that refers to objects of unknown categories

9Citations
Citations of this article
121Readers
Mendeley users who have this article in their library.

Abstract

Zero-shot learning in Language & Vision is the task of correctly labelling (or naming) objects of novel categories. Another strand of work in L&V aims at pragmatically informative rather than “correct” object descriptions, e.g. in reference games. We combine these lines of research and model zero-shot reference games, where a speaker needs to successfully refer to a novel object in an image. Inspired by models of “rational speech acts”, we extend a neural generator to become a pragmatic speaker reasoning about uncertain object categories. As a result of this reasoning, the generator produces fewer nouns and names of distractor categories as compared to a literal speaker. We show that this conversational strategy for dealing with novel objects often improves communicative success, in terms of resolution accuracy of an automatic listener.

References Powered by Scopus

ImageNet: A Large-Scale Hierarchical Image Database

50503Citations
N/AReaders
Get full text

Microsoft COCO: Common objects in context

28766Citations
N/AReaders
Get full text

Show and tell: A neural image caption generator

4679Citations
N/AReaders
Get full text

Cited by Powered by Scopus

Decoding methods in neural language generation: A survey

15Citations
N/AReaders
Get full text

Linguistic issues behind visual question answering

13Citations
N/AReaders
Get full text

Learning to Mediate Disparities Towards Pragmatic Communication

7Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Zarrieß, S., & Schlangen, D. (2020). Know what you don’t know: Modeling a pragmatic speaker that refers to objects of unknown categories. In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (pp. 654–659). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/p19-1063

Readers over time

‘19‘20‘21‘22‘23‘24‘2508162432

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 38

68%

Researcher 12

21%

Professor / Associate Prof. 3

5%

Lecturer / Post doc 3

5%

Readers' Discipline

Tooltip

Computer Science 51

80%

Linguistics 8

13%

Neuroscience 3

5%

Engineering 2

3%

Save time finding and organizing research with Mendeley

Sign up for free
0