Bilingual sentence alignment: balancing robustness and accuracy

BILINGUAL SENTENCE ALIGNMENT: BALANCING ROBUSTNESS AND ACCURACY Michel Simard Pierre Plamondon simard@citi.doc.ca plamondo@citi.doc.ca Centre for Information Technology Innovation (CITI) 1575 Chomedey Blvd. Laval (Quebec) CANADA H7V 2X2 Abstract Sentence alignment is the problem of making explicit the relations that exist between the sentences of two texts that are known to be mutual translations. Automatic sentence alignment methods typically face two kinds of difficulties. First, there is the question of robustness. In real life, discrepancies between the source-text and its translation are quite common: differences in layout, omissions, inversions, etc. Sentence alignment programs must be ready to deal with such phenomena. Then, there is the question of accuracy. Even when translations are "clean", alignment is still not a trivial matter: some decisions are hard to make, even for humans. We report here on the current state of our ongoing efforts to produce a sentence alignment program that is both robust and accurate. The method that we propose relies on two new alignment engines, and combines the robustness of so-called "character-based" methods with the accuracy of stochastic translation models. Experimental results are presented, that demonstrate the method's effectiveness, and highlight where problems remain to be solved. Introduction The bitext correspondence problem (BCP) can be loosely described as that of making explicit the relations that exist between two texts that are known to be mutual translations. The result of this operation can take many forms, but the output of most existing bitext correspondence methods falls into one of two categories: • An alignment is a parallel segmentation of the two texts, typically into small logical units such as sentences, such that the nth segment of the first text and the nth segment of the second are mutual translations. • A bitext map is a set of pairs (x, y), where x and y refer to precise locations in the first and second texts respectively, with the intention of denoting portions of the texts that correspond to one another. This is illustrated in figure 1. Bitext correspondences are of vital interest to anyone who wishes to exploit existing translations as an active source of information. Which is best between an alignment and a bitext map usually depends on the intended application. By definition, an alignment covers the totality of the bitext. In this sense, it is both exhaustive and exact: for each segment of text, it says something like "the translation of this segment is exactly that segment". The same cannot be said of bitext maps. Some methods, such as those proposed by Church [5] or Fung and McKeown [7], produce approximate maps (i.e. not exact), that say something like "The translation of the text around this point is somewhere around that point". Other methods, such as those 135 proposed by Dagan et al. [6] or Melamed [12] produce maps that are exact ("the translation of the object at position x is the object at position y") but not exhaustive. On the other hand, what they lack in exactness or exhaustiveness, bitext map usually …

Content truncated to 500 words