A multitask objective to inject lexical contrast into distributional semantics

Abstract

Distributional semantic models have trou-ble distinguishing strongly contrasting words (such as antonyms) from highly compatible ones (such as synonyms), be-cause both kinds tend to occur in similar contexts in corpora. We introduce the mul-titask Lexical Contrast Model (mLCM), an extension of the effective Skip-gram method that optimizes semantic vectors on the joint tasks of predicting corpus contexts and making the representations of WordNet synonyms closer than that of matching WordNet antonyms. mLCM outperforms Skip-gram both on general semantic tasks and on synonym/antonym discrimination, even when no direct lex-ical contrast information about the test words is provided during training. mLCM also shows promising results on the task of learning a compositional negation oper-ator mapping adjectives to their antonyms.