Reducing Phrase-based SMT Translation Tables By Phrase Pair Boundary Scope

[[abstract]]在以詞組為本的統計式機器翻譯系統 (Phrase-Based Statistical Machine Translation, Phrase-Based SMT) 中，詞組翻譯對照表 (Phrase Translation Table) 裡的詞組對(Phrase Pairs) 一般皆遵從直覺的經驗法則方式 (Heuristic) 來進行擷取。也就是說，最常使用的方法主要是以最佳的詞彙對應 (word alignment) 為本，透過heuristic的方法找出可能的Phrase Pairs。以這種方法擷取出來的Phrases被視為是連續的字彙序列(Sequences of Consecutive Words)，故在缺乏客觀的詞組分斷 (Phrase Segmentation) 標準下，符合規則但雜亂無章的Phrase Pairs會大量產生，導致Phrase Translation Table過於龐大而降低翻譯的品質 (BLEU score)。 為此，本文提出以詞組對邊界範圍 (Phrase Pair Boundary Scope，PPBS) 來縮減之Phrase Translation Table之方法，輔助Phrase-Based SMT篩選Phrase Pair 。在抓取合適的Phrase Pair之前，本法使用EM演算法來評估所有可能的Phrase segments的機率，並且使用此機率來對Sentence做Phrase分斷處理，找出最佳的Phrase Boundaries。之後，在傳統SMT (如Moses)產生Phrase Pairs過程中，加入PPBS條件，來限制每組Phrase Pair是否該被擷取，使得不符合PPBS條件的Phrase Pair不會被擷取到Phrase Translation Table中，來達到減少Phrase Translation Table的大小，並且提昇翻譯的品質。[[abstract]]In Phrase-Based Statistical Machine Translation (Phrase-Based SMT) systems, phrase pairs of the phrase translation table are generally extracted heuristically. That is, the most possible phrase pairs are extracted based on word alignment results. Such phrases are no more than sequences of consecutive words, without objective phrase segmentation criteria, a lot of messy phrase pairs will be generated, leading to extremely huge phrase translation table and diminish translation quality (in BLEU score). To overcome the problems introduced by the messy and huge phrase translation table, this article proposed to use Phrase Pair Boundary Scope (PPBS) information to assist phrase translation table reduction. First of all, we use an EM algorithm to estimate the probabilities of all possible phrase segmentations, which will identify the best phrase boundaries of the source and target sentences. We then check phrase pairs against the best phrase boundaries to prevent the extraction of inappropriate phrases into the phrase translation table. Through this approach, the translation quality may be improved in some cases and the size of phrase translation table will be reduced.[[note]]碩