Improving sparse word representations with distributional inference for semantic composition

10Citations
Citations of this article
83Readers
Mendeley users who have this article in their library.

Abstract

Distributional models are derived from co-occurrences in a corpus, where only a small proportion of all possible plausible co-occurrences will be observed. This results in a very sparse vector space, requiring a mechanism for inferring missing knowledge. Most methods face this challenge in ways that render the resulting word representations uninterpretable, with the consequence that semantic composition becomes hard to model. In this paper we explore an alternative which involves explicitly inferring unobserved co-occurrences using the distributional neighbourhood. We show that distributional inference improves sparse word representations on several word similarity benchmarks and demonstrate that our model is competitive with the state-of-the-art for adjective-noun, noun-noun and verb-object compositions while being fully interpretable.

Cite

CITATION STYLE

APA

Kober, T., Weeds, J., Reffin, J., & Weir, D. (2016). Improving sparse word representations with distributional inference for semantic composition. In EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings (pp. 1691–1702). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/d16-1175

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free