Marker-based Filtering of Bilingual Phrase Pairs for SMT

State-of-the-art statistical machine transla-tion systems make use of a large trans-lation table obtained after scoring a set of bilingual phrase pairs automatically ex-tracted from a parallel corpus. The num-ber of bilingual phrase pairs extracted from a pair of aligned sentences grows expo-nentially as the length of the sentences in-creases; therefore, the number of entries in the phrase table used to carry out the translation may become unmanageable, es-pecially when online, ‘on demand ’ trans-lation is required in real time. We de-scribe the use of closed-class words to fil-ter the set of bilingual phrase pairs ex-tracted from the parallel corpus by tak-ing into account the alignment information and the type of the words involved in the alignments. On four European language pairs, we show that our simple yet novel approach can filter the phrase table by up to a third yet still provide competitive results compared to the baseline. Furthermore, it provides a nice balance between the un-filtered approach and pruning using stop words, where the deterioration in transla-tion quality is unacceptably high.