Unsupervised post-processing of word vectors via conceptor negation

Abstract

Word vectors are at the core of many natural language processing tasks. Recently, there has been interest in postprocessing word vectors to enrich their semantic information. In this paper, we introduce a novel word vector postprocessing technique based on matrix conceptors (Jaeger 2014), a family of regularized identity maps. More concretely, we propose to use conceptors to suppress those latent features of word vectors having high variances. The proposed method is purely unsupervised: it does not rely on any corpus or external linguistic database. We evaluate the post-processed word vectors on a battery of intrinsic lexical evaluation tasks, showing that the proposed method consistently outperforms existing state-of-the-art alternatives. We also show that post-processed word vectors can be used for the downstream natural language processing task of dialogue state tracking, yielding improved results in different dialogue domains.