Retrofitting word vectors to semantic lexicons

Vector space word representations are typically learned using only co-occurrence statistics from text corpora. Although such statistics are informative, they disre-gard easily accessible (and often carefully curated) information archived in se-mantic lexicons such as WordNet, FrameNet, and the Paraphrase Database. This paper proposes a technique to leverage both distributional and lexicon-derived ev-idence to obtain better representations. We run belief propagation on a word type graph constructed from word similarity information from lexicons to encourage connected (related) words to have similar representations, and also to be close to the unsupervised vectors. Evaluated on a battery of standard lexical semantic evaluation tasks in several languages, using several different underlying word vec-tor models, we obtain substantially improved vectors and consistently outperform existing approaches of incorporating semantic knowledge in word vectors.