Exploring Continual Learning of Compositional Generalization in NLI

Abstract

Compositional Natural Language Inference (NLI) has been explored to assess the true abilities of neural models to perform NLI. Yet, current evaluations assume models to have full access to all primitive inferences in ad-vance, in contrast to humans that continuously acquire inference knowledge. In this paper, we introduce the Continual Compositional Generalization in Inference (C2 Gen NLI) chal-lenge, where a model continuously acquires knowledge of constituting primitive inference tasks as a basis for compositional in-ferences. We explore how continual learning affects compositional generalization in NLI, by designing a continual learning setup for compositional NLI inference tasks. Our ex-periments demonstrate that models fail to compositionally generalize in a continual sce-nario. To address this problem, we first benchmark various continual learning algo-rithms and verify their efficacy. We then further analyze C2 Gen, focusing on how to order primitives and compositional inference types, and examining correlations between subtasks. Our analyses show that by learning subtasks continuously while observing their dependencies and increasing degrees of difficulty, continual learning can enhance composition generalization ability.1.