A comparison of character and word embeddings in bidirectional LSTMs for POS tagging in Italian

Abstract

Word representations are mathematical items capturing a word’s meaning and its grammatical properties in a machine-readable way. They map each word into equivalence classes including words sharing similar properties. Word representations can be obtained automatically by using unsupervised learning algorithms that rely on the distributional hypothesis, stating that the meaning of a word is strictly connected to its context in terms of surrounding words. This assessed notion of context has been recently reconsidered in order to include both distributional and morphological features of a word in terms of characters co-occurrence. This approach has evidenced very promising results, especially in NLP tasks, e.g, POS Tagging, where the representation of the so-called Out of Vocabulary (OOV) words represents a partially solved issue. This work is intended to face the problem of representing OOV words for a POS Tagging task, contextualized to the Italian language. Potential benefits and drawbacks of adopting a Bidirectional Long Short Term Memory (bi-LSTM) fed with a joint character and word embeddings representation to perform POS Tagging also considering OOV words have been investigated. Furthermore, experiments have been performed and discussed by estimating qualitative and quantitative indicators, and, thus, suggesting some possible future direction of the investigation.