Leveraging large language models for word sense disambiguation

Abstract

Natural language processing (NLP) is difficult because human language contains ambiguity. The same word can have a different meaning depending on the context and may result in different interpretations given biases held by a NLP technique. Correctly interpreting this ambiguity is not simply an important task in its own right but is a key enabler to major NLP activities such as machine translation and question answering. This research proposes three techniques to evaluate a large language models’(LLMs) ability to perform word sense disambiguation (WSD) and explores the efficacy of seven generative LLMs. The first technique assesses whether LLMs can, given a context sentence, select the correct word sense from a menu of options. The second asks LLMs, without options provided, to state whether or not a provided word sense is correct. The third technique presents the LLMs with context and an unseen word, assessing whether the LLMs can infer from context the sense of a word that it has not seen during training. Results demonstrate a strong relationship between model size and performance. Applications of WSD are demonstrated as part of an information extraction pipelines supporting sentiment analysis and as part of an LLM-evaluation suite to support machine learning operations.