A Binarized Neural Network Joint Model for Machine Translation

The neural network joint model (NNJM), which augments the neural network lan-guage model (NNLM) with an m-word source context window, has achieved large gains in machine translation accuracy, but also has problems with high normalization cost when using large vocabularies. Train-ing the NNJM with noise-contrastive es-timation (NCE), instead of standard maxi-mum likelihood estimation (MLE), can re-duce computation cost. In this paper, we propose an alternative to NCE, the bina-rized NNJM (BNNJM), which learns a bi-nary classifier that takes both the context and target words as input, and can be ef-ficiently trained using MLE. We compare the BNNJM and NNJM trained by NCE on various translation tasks.