Effective bilingual constraints for semisupervised learning of named entity recognizers

Most semi-supervised methods in Natural Language Process-ing capitalize on unannotated resources in a single language; however, information can be gained from using parallel re-sources in more than one language, since translations of the same utterance in different languages can help to disam-biguate each other. We demonstrate a method that makes effective use of vast amounts of bilingual text (a.k.a. bi-text) to improve monolingual systems. We propose a factored probabilistic sequence model that encourages both cross-language and intra-document consistency. A simple Gibbs sampling algorithm is introduced for performing approximate inference. Experiments on English-Chinese Named Entity Recognition (NER) using the OntoNotes dataset demonstrate that our method is significantly more accurate than state-of-the-art monolingual CRF models in a bilingual test setting. Our model also improves on previous work by Burkett et al. (2010), achieving a relative error reduction of 10.8 % and 4.5 % in Chinese and English, respectively. Furthermore, by annotating a moderate amount of unlabeled bi-text with our bilingual model, and using the tagged data for uptraining, we achieve a 9.2 % error reduction in Chinese over the state-of-the-art Stanford monolingual NER system