Manifold Learning-based Word Representation Refinement Incorporating Global and Local Information

3Citations
Citations of this article
60Readers
Mendeley users who have this article in their library.

Abstract

Recent studies show that word embedding models often underestimate similarities between similar words and overestimate similarities between distant words. This results in word similarity results obtained from embedding models inconsistent with human judgment. Manifold learning-based methods are widely utilized to refine word representations by re-embedding word vectors from the original embedding space to a new refined semantic space. These methods mainly focus on preserving local geometry information through performing weighted locally linear combination between words and their neighbors twice. However, these reconstruction weights are easily influenced by different selections of neighboring words and the whole combination process is time-consuming. In this paper, we propose two novel word representation refinement methods leveraging isometry feature mapping and local tangent space respectively. Unlike previous methods, our first method corrects pre-trained word embeddings by preserving global geometry information of all words instead of local geometry information between words and their neighbors. Our second method refines word representations by aligning original and refined embedding spaces based on local tangent space instead of performing weighted locally linear combination twice. Experimental results obtained from standard semantic relatedness and semantic similarity tasks show that our methods outperform various state-of-the-art baselines for word representation refinement.

Cite

CITATION STYLE

APA

Zhao, W., Zhou, D., Li, L., & Chen, J. (2020). Manifold Learning-based Word Representation Refinement Incorporating Global and Local Information. In COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference (pp. 3401–3412). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2020.coling-main.301

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free