Numbers have unique characteristics to words. Teaching models to understand numbers in text is an open-ended research question. Instead of discussing the required calculation skills, this paper focuses on a more fundamental topic: understanding numerals. We point out that innumeracy—the inability to handle basic numeral concepts—exists in most pretrained language models (LMs), and we propose a method to solve this issue by exploring the notation of numbers. Further, we discuss whether changing notation and pre-finetuning along with the comparing-number task can improve performance in three benchmark datasets containing quantitative-related tasks. The results of this study indicate that input reframing and the proposed pre-finetuning task is useful for RoBERTa.
CITATION STYLE
Chen, C. C., Takamura, H., Kobayashi, I., & Miyao, Y. (2023). Improving Numeracy by Input Reframing and Quantitative Pre-Finetuning Task. In EACL 2023 - 17th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2023 (pp. 69–77). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.findings-eacl.4
Mendeley helps you to discover research relevant for your work.