Probing Physical Reasoning with Counter-Commonsense Context

1Citations
Citations of this article
6Readers
Mendeley users who have this article in their library.

Abstract

In this study, we create a CConS (Counter-commonsense Contextual Size comparison) dataset to investigate how physical commonsense affects the contextualized size comparison task; the proposed dataset consists of both contexts that fit physical commonsense and those that do not. This dataset tests the ability of language models to predict the size relationship between objects under various contexts generated from our curated noun list and templates. We measure the ability of several masked language models and generative models. The results show that while large language models can use prepositions such as “in” and “into” in the provided context to infer size relationships, they fail to use verbs and thus make incorrect judgments led by their prior physical commonsense.

Cite

CITATION STYLE

APA

Kondo, K., Sugawara, S., & Aizawa, A. (2023). Probing Physical Reasoning with Counter-Commonsense Context. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Vol. 2, pp. 603–612). Association for Computational Linguistics (ACL). https://doi.org/10.5715/jnlp.30.1261

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free