A unified learning framework of skip-grams and global vectors

Abstract

Log-bilinear language models such as SkipGram and GloVe have been proven to capture high quality syntactic and seman-tic relationships between words in a vector space. We revisit the relationship between SkipGram and GloVe models from a ma-chine learning viewpoint, and show that these two methods are easily merged into a unified form. Then, by using the unified form, we extract the factors of the config-urations that they use differently. We also empirically investigate which factor is re-sponsible for the performance difference often observed in widely examined word similarity and analogy tasks.