Online Learning of Interpretable Word Embeddings

Word embeddings encode semantic mean-ings of words into low-dimension word vectors. In most word embeddings, one cannot interpret the meanings of specific dimensions of those word vectors. Non-negative matrix factorization (NMF) has been proposed to learn interpretable word embeddings via non-negative constraints. However, NMF methods suffer from scale and memory issue because they have to maintain a global matrix for learning. To alleviate this challenge, we propose on-line learning of interpretable word embed-dings from streaming text data. Exper-iments show that our model consistently outperforms the state-of-the-art word em-bedding methods in both representation a-bility and interpretability. The source code of this paper can be obtained fro