Stacking of BERT and CNN Models for Arabic Word Sense Disambiguation

Abstract

We propose a new approach for Arabic Word Sense Disambiguation (AWSD) by hybridization of single-layer Convolutional Neural Network (CNN) with contextual representation (BERT). WSD is the task of automatically detecting the correct meaning of a word used in a given context. WSD can be performed as a classification task, and the context is generally a short sentence. Kim [26] proved that combining a CNN with an RNN (recurrent neural network) provides a good result for text classification. Here, we use a concatenation of BERT models as a word embedding to get simultaneously the target and context representation. Our approach improves the performance of WSD in Arabic languages. The experimental results show that our model outperforms the state-of-the-art approaches and improves the accuracy of 96.42% on the Arabic WordNet dataset.