Continuous space language models for statistical machine translation

Statistical machine translation systems are based on one or more translation mod-els and a language model of the target language. While many different trans-lation models and phrase extraction al-gorithms have been proposed, a standard word n-gram back-off language model is used in most systems. In this work, we propose to use a new sta-tistical language model that is based on a continuous representation of the words in the vocabulary. A neural network is used to perform the projection and the proba-bility estimation. We consider the trans-lation of European Parliament Speeches. This task is part of an international evalua-tion organized by the TC-STAR project in 2006. The proposed method achieves con-sistent improvements in the BLEU score on the development and test data. We also present algorithms to improve the estimation of the language model proba-bilities when splitting long sentences into shorter chunks.