Decoupled word embeddings using latent topics

Abstract

In this paper, we propose decoupled word embeddings (DWE) as a universal word representation that covers multiple senses of words. Toward this goal, our model represents each word as a combination of multiple word vectors that are associated with latent topics. Specifically, we decompose a word vector into multiple word vectors for multiple senses, according to the topic weight obtained from pre-trained topic models. Although this dynamic word representation is simple, the proposed model can leverage both local and global contexts. Through extensive experiments, including qualitative and quantitative analyses, we demonstrate that the proposed model is comparable to or better than state-of-the-art word embedding models. The code is publicly available at https://github.com/righ120/DWE.