Contextualized context2vec

Abstract

Lexical substitution ranks substitution candi- dates from the viewpoint of paraphrasability for a target word in a given sentence. There are two major approaches for lexical substitu- tion: (1) generating contextualized word em- beddings by assigning multiple embeddings to one word and (2) generating context embed- dings using the sentence. Herein we propose a method that combines these two approaches to contextualize word embeddings for lexi- cal substitution. Experiments demonstrate that our method outperforms the current state-of- the-art method. We also create CEFR-LP, a new evaluation dataset for the lexical substi- tution task. It has a wider coverage of sub- stitution candidates than previous datasets and assigns English proficiency levels to all target words and substitution candidates.