Can we neglect function words in word embedding?

Abstract

Distributed representation is the most popular way to capture semantic and syntactic features recently, and it has been widely used in various natural language processing tasks. Function words express a grammatical or structural relationship with other words in a sentence. However, previous works merely considered that function words are equal to content words or neglected function words, there is no experimental analyses about function words. In this paper, we explored the effect of function words on word embedding with a word analogy reasoning task and a paraphrase identification task. The results show that neglecting function words has different effects on syntactic and semantic related tasks, with an increase or a decrease in accuracy, moreover, the model of training word embeddings does also matter.