Meaning as Use: From Wittgenstein to Google’s Word2vec

Abstract

Modern natural language processingNatural language processing (NLP) systems are based on neural networks that learn concept representation directly from data. In such systems, concepts are represented by real number vectors, with the background idea that mapping words into vectors should take into account the context of their use. The idea is present in Wittgenstein'sWittgenstein both early and late works, as well as in contemporary general linguistics, especially in the works of FirthFirth. In this article, we investigate the relevance of Wittgenstein'sWittgenstein and Firth'sFirth ideas for the development of Word2vecWord2vec, a word vector representation used in a machine translationMachine translation model developed by Google. We argue that one of the chief differences between Wittgenstein'sWittgenstein and Firth'sFirth approaches to the word meaning, compared to the one applied in Word2vecWord2vec, lies in the fact that, although all of them emphasise the importance of context, its scope is differently understood.