SUT at SemEval-2023 Task 1: Prompt Generation for Visual Word Sense Disambiguation

Abstract

Visual Word Sense Disambiguation (V-WSD) identifies the correct visual sense of a multisense word in a specific context. This can be challenging as images may need to provide additional context and words may have multiple senses. A proper V-WSD system can benefit applications like image retrieval and captioning. This paper proposes a Prompt Generation approach to solve this challenge. This approach improves the robustness of language-image models like CLIP to contextual ambiguities and helps them better correlate between textual and visual contexts of different senses of words.