kNN-CM: A Non-parametric Inference-Phase Adaptation of Parametric Text Classifiers

Abstract

Semi-parametric models exhibit the properties of both parametric and non-parametric modeling and have been shown to be effective in the next-word prediction language modeling task. However, there is a lack of studies on the text-discriminating properties of such models. We propose an inference-phase approach-kNearest Neighbor Classification Model (kNNCM)-that enhances the capacity of a pre-trained parametric text classifier by incorporating a simple neighborhood search through the representation space of (memorized) training samples. The final class prediction of kNNCM is based on the convex combination of probabilities obtained from kNN search and prediction of the classifier. Our experiments show consistent performance improvements on eight SuperGLUE tasks, three adversarial natural language inference (ANLI) datasets, 11 question-answering (QA) datasets, and two sentiment classification datasets. The source code of the proposed approach is available at https://github.com/Bhardwaj-Rishabh/kNN-CM.