Reducing Asymmetry between language-pairs to Improve Alignment and Translation Quality

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1346–1350, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Reducing Asymmetry between language-pairs to Improve Alignment and Translation Quality Rashmi Gangadharaiah IBM Research, India rashgang@in.ibm.com Abstract This paper presents a novel method to re- move asymmetry between the source and the target languages thereby improving alignment and machine translation (MT) quality. Some words in the source lan- guage are redundant for MT tasks but nec- essary for the source sentence to be gram- matical. This paper proposes a method to automatically detect such words. In addition, constraints under which these words should or should not be removed are extracted automatically from the tar- get language. A lattice scheme is used for test sentences to provide alternate paths with and without removal of these words. Such a constraint-based removal technique gives a significant improvement (p < 0.001) of 5.29 BLEU points over the baseline Phrase-based MT system for the English-Hindi language-pair. 1 Introduction Different languages express the same piece of in- formation with different number of words. As a result, in many language-pairs, not all source words have correspondences in the target half of the sentence-pair. The aligner is expected to align these redundant source words to an empty word (“NULL”) in the target sentence. However, in data-sparse conditions, this is not perfectly learnt. The IBM models in GIZA++ (Och and Ney, 2003) align each source word to exactly one target word and a target word can be aligned to multiple source words. Currently in most MT systems, this limitation in alignment is overcome by aligning the data bi-directionally (Koehn et al., 2003) and later combining the resulting alignments. Inspite of using information from bi-directional align- ments, due to the noisy nature and limitations in the amount of parallel data available, low qual- ity word-alignments are obtained. To illustrate this, consider the top ten maximum likelihood lex- ical translation-table entries for the English source words, “the” and “india” obtained after combin- ing the bi-directional alignments with the English- Hindi language-pair in Figures 3A and 3B. For the sake of clarity, the actual English translations of the Hindi words (with case-markers transliterated) are displayed instead of the Hindi words them- selves. As seen, the probability mass is used up by many other Hindi words that are not the right translations of the source words. Also, in Fig- ure 3B, “india” has a higher probability of being aligned to “NULL” than to its actual translation. The most common errors in data that result in low quality alignments and translations are: 1) Property of a language-pair: Certain words are necessary to be present in a sentence for the sentence to be grammatical but unnecessary for MT. For example, the in “the government was highly perturbed by his activities” has no corre- spondence in Hindi. For MT, these words can hurt the performance of the aligner (illustrated in Fig- ure 1) and translation quality (Section 4). This is true for many …

Content truncated to 500 words