First Steps Towards Coverage-Based Sentence Alignment

First Steps Towards Coverage-Based Sentence Alignment Lu´ıs Gomes1,2, Gabriel P. Lopes 1,2 1NOVA LINCS, Faculdade de Ciˆencias e Tecnologia, Universidade Nova de Lisboa, Portugal 2ISTRION BOX Translation and Revision, Lda, Portugal luis.gomes@istrionbox.com gabriel.lopes@istrionbox.com Abstract In this paper, we introduce a coverage-based scoring function that discriminates between parallel and non-parallel sentences. When plugged into Bleualign, a state-of-the-art sentence aligner, our function improves both precision and recall of alignments over the orig- inally proposed BLEU score. Furthermore, since our scoring function uses Moses phrase tables directly we avoid the need to translate the texts to be aligned, which is time-consuming and a potential source of alignment errors. Keywords: Phrase-Table Coverage, Sentence Alignment, Language Independent 1. Introduction Sentence alignment is the task of finding corresponding sentences between texts that are translations of each other (parallel texts). It is a fundamental task for corpora-based approaches to Machine Translation, such as Statistical Ma- chine Translation and Example-Based Machine Transla- tion, as well as for creating Translation Memories to be used in Computer-Assisted Translation. Some parallel texts such as those from the European Parlia- ment and the Canadian Hansard are relatively easy to align because the translations are very clean, i.e. there is almost a one-to-one correspondence between sentences in both lan- guages. Furthermore, those texts have markup that can be used as anchor points to constrain the alignment, allowing accuracies above 95% even with alignment methods based exclusively on sentence length proportionality, such as the ones proposed by Brown et al. (1991) and Gale and Church (1993) (the former measures sentence length in terms of to- kens and the latter in terms of characters). However, for texts available in less-friendly formats, such as PDF, from which we cannot avoid extracting some noise intermixed with the text (such as figure and table captions, page headers and footers, etc) we need more robust aligners that take into account the actual text within sentences and not only their lengths. In this paper, we improve over Bleualign (Sennrich and Volk, 2010), a state-of-the-art sentence aligner with top- performance on noisy texts (Sennrich and Volk, 2010; Abdul-Rauf et al., 2012). Our main contribution is a new scoring function that dis- criminates parallel and non-parallel sentences based on the ratio of text covered by bilingual phrase-pairs from a Moses phrase table (Koehn et al., 2007). 2. Previous Work The general idea of Bleualign is to automatically translate one of the texts and then align the translation with the other text using the sentence-wise BLEU score (Papineni et al., 2002) as indicator of sentence similarity. Besides the good performance on noisy texts, Bleualign appealed to us because, in a way, it takes advantage of previously acquired translation knowledge that is encoded within the MT system used to translate the texts to be aligned. By contrast, other aligners such as the Microsoft Bilingual Aligner (Moore, 2002), Hunalign (Varga et al., 2005) and Gargantua (Braune and Fraser, 2010) are au- tonomous and automatically infer a word-based translation lexicon from the texts being aligned as …

Content truncated to 500 words