Disambiguating Grammatical Number and Gender With BERT

Abstract

Accurately dealing with any type of ambiguity is a major task in Natural Language Processing, with great advances recently reached due to the development of context dependent language models and the use of word or sentence embeddings. In this context, our work aimed at determining how the popular language representation model BERT handles ambiguity of nouns in grammatical number and gender in different languages. This work shows that models trained on one specific language achieve better results for the disambiguation process than multilingual models. Also, ambiguity is generally better dealt with in grammatical number than it is in grammatical gender, reaching greater distance values from one to another in direct comparisons of individual word sense embeddings. The overall results show also that the amount of data needed for training monolingual models as well as application should not be underestimated.