Partitioning Parallel Documents Using Binary Segmentation

Proceedings of the Workshop on Statistical Machine Translation, pages 78–85, New York City, June 2006. c⃝2006 Association for Computational Linguistics Partitioning Parallel Documents Using Binary Segmentation Jia Xu and Richard Zens and Hermann Ney Chair of Computer Science 6 Computer Science Department RWTH Aachen University D-52056 Aachen Germany {xujia,zens,ney}@cs.rwth-aachen.de Abstract In statistical machine translation, large numbers of parallel sentences are required to train the model parameters. However, plenty of the bilingual language resources available on web are aligned only at the document level. To exploit this data, we have to extract the bilingual sentences from these documents. The common method is to break the doc- uments into segments using predefined anchor words, then these segments are aligned. This approach is not error free, incorrect alignments may decrease the translation quality. We present an alternative approach to ex- tract the parallel sentences by partitioning a bilingual document into two pairs. This process is performed recursively until all the sub-pairs are short enough. In experiments on the Chinese-English FBIS data, our method was capable of producing translation results comparable to those of a state-of-the-art sentence aligner. Using a combination of the two approaches leads to better translation per- formance. 1 Introduction Current statistical machine translation systems use bilingual sentences to train the parameters of the translation models. The exploitation of more bilin- gual sentences automatically and accurately as well as the use of these data with the limited computa- tional requirements become crucial problems. The conventional method for producing parallel sentences is to break the documents into sentences and to align these sentences using dynamic program- ming. Previous investigations can be found in works such as (Gale and Church, 1993) and (Ma, 2006). A disadvantage is that only the monotone sentence alignments are allowed. Another approach is the binary segmentation method described in (Simard and Langlais, 2003), (Xu et al., 2005) and (Deng et al., 2006), which separates a long sentence pair into two sub-pairs re- cursively. The binary reordering in alignment is al- lowed but the segmentation decision is only opti- mum in each recursion step. Hence, a combination of both methods is ex- pected to produce a more satisfying result. (Deng et al., 2006) performs a two-stage procedure. The documents are first aligned at level using dynamic programming, the initial alignments are then refined to produce shorter segments using binary segmen- tation. But on the Chinese-English FBIS training corpus, the alignment accuracy and recall are lower than with Champollion (Ma, 2006). We refine the model in (Xu et al., 2005) using a log-linar combination of different feature func- tions and combine it with the approach of (Ma, 2006). Here the corpora produced using both ap- proaches are concatenated, and each corpus is as- signed a weight. During the training of the word alignment models, the counts of the lexicon entries 78 are linear interpolated using the corpus weights. In the experiments on the Chinese-English FBIS cor- pus the translation performance is improved by 0.4% of the BLEU score compared to …

Content truncated to 500 words