Social IQA: Commonsense reasoning about social interactions

326Citations
Citations of this article
276Readers
Mendeley users who have this article in their library.

Abstract

We introduce SOCIAL IQA, the first large-scale benchmark for commonsense reasoning about social situations. SOCIAL IQA contains 38,000 multiple choice questions for probing emotional and social intelligence in a variety of everyday situations (e.g., Q: “Jordan wanted to tell Tracy a secret, so Jordan leaned towards Tracy. Why did Jordan do this?” A: “Make sure no one else could hear”). Through crowdsourcing, we collect commonsense questions along with correct and incorrect answers about social interactions, using a new framework that mitigates stylistic artifacts in incorrect answers by asking workers to provide the right answer to a different but related question. Empirical results show that our benchmark is challenging for existing question-answering models based on pretrained language models, compared to human performance (>20% gap). Notably, we further establish SOCIAL IQA as a resource for transfer learning of commonsense knowledge, achieving state-of-the-art performance on multiple commonsense reasoning tasks (Winograd Schemas, COPA).

References Powered by Scopus

Does the autistic child have a "theory of mind" ?

5468Citations
N/AReaders
Get full text

SQuad: 100,000+ questions for machine comprehension of text

3973Citations
N/AReaders
Get full text

Very large and huge effect sizes

1713Citations
N/AReaders
Get full text

Cited by Powered by Scopus

The neural architecture of language: Integrative modeling converges on predictive processing

243Citations
N/AReaders
Get full text

How can we know when language models know? On the calibration of language models for question answering

203Citations
N/AReaders
Get full text

Generated Knowledge Prompting for Commonsense Reasoning

134Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Sap, M., Rashkin, H., Chen, D., Le Bras, R., & Choi, Y. (2019). Social IQA: Commonsense reasoning about social interactions. In EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference (pp. 4463–4473). Association for Computational Linguistics. https://doi.org/10.18653/v1/d19-1454

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 100

79%

Researcher 15

12%

Professor / Associate Prof. 7

6%

Lecturer / Post doc 4

3%

Readers' Discipline

Tooltip

Computer Science 122

84%

Linguistics 12

8%

Engineering 9

6%

Social Sciences 3

2%

Save time finding and organizing research with Mendeley

Sign up for free