We propose a novel challenge for large language models: ellipsis-dependent reasoning. We define several structures of paired examples, where an ellipsis example is matched to its non-ellipsis counterpart, and a question is posed which requires resolution of the ellipsis. Test results show that the best models perform well on non-elliptical examples but struggle with all but the simplest ellipsis structures.
CITATION STYLE
Hardt, D. (2023). Ellipsis-Dependent Reasoning: a New Challenge for Large Language Models. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Vol. 2, pp. 39–47). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.acl-short.4
Mendeley helps you to discover research relevant for your work.