Cost Optimization for Crowdsourcing Translation

Crowdsourcing makes it possible to create translations at much lower cost than hiring professional translators. However, it is still expensive to obtain the millions of transla-tions that are needed to train statistical ma-chine translation systems. We propose two mechanisms to reduce the cost of crowdsourc-ing while maintaining high translation quality. First, we develop a method to reduce redun-dant translations. We train a linear model to evaluate the translation quality on a sentence-by-sentence basis, and fit a threshold between acceptable and unacceptable translations. Un-like past work, which always paid for a fixed number of translations for each source sen-tence and then chose the best from them, we can stop earlier and pay less when we receive a translation that is good enough. Second, we introduce a method to reduce the pool of translators by quickly identifying bad transla-tors after they have translated only a few sen-tences. This also allows us to rank translators, so that we re-hire only good translators to re-duce cost.