Splitting Input Sentence for Machine Translation Using Language Model with Sentence Similarity

Splitting Input Sentence for Machine Translation Using Language Model with Sentence Similarity Takao Doi Eiichiro Sumita ATR Spoken Language Translation Research Laboratories 2-2-2 Hikaridai, Kansai Science City, Kyoto, 619-0288 Japan {takao.doi, eiichiro.sumita}@atr.jp Abstract In order to boost the translation quality of corpus-based MT systems for speech transla- tion, the technique of splitting an input sen- tence appears promising. In previous re- search, many methods used N-gram clues to split sentences. In this paper, to supplement N-gram based splitting methods, we introduce another clue using sentence similarity based on edit-distance. In our splitting method, we generate candidates for sentence splitting based on N-grams, and select the best one by measuring sentence similarity. We conducted experiments using two EBMT systems, one of which uses a phrase and the other of which uses a sentence as a translation unit. The translation results on various conditions were evaluated by objective measures and a subjec- tive measure. The experimental results show that the proposed method is valuable for both systems. 1 Introduction We are exploring methods to boost the transla- tion quality of corpus-based Machine Translation (MT) systems for speech translation. Among them, the technique of splitting an input sentence and translating the split sentences appears promis- ing (Doi and Sumita, 2003). An MT system sometimes fails to translate an input correctly. Such a failure occurs particu- larly when an input is long. In such a case, by splitting the input, translation may be successfully performed for each portion. Particularly in a di- alogue, sentences tend not to have complicated nested structures, and many long sentences can be split into mutually independent portions. There- fore, if the splitting positions and the translations of the split portions are adequate, the possibility that the arrangement of the translations can pro- vide an adequate translation of the complete in- put is relatively high. For example, the input sen- tence, ”This is a medium size jacket I think it’s a good size for you try it on please” 1 can be split into three portions, ”This is a medium size jacket”, ”I think it’s a good size for you” and ”try it on please”. In this case, translating the three portions and arranging the results in the same order give us the translation of the input sentence. In previous research on splitting sentences, many methods have been based on word-sequence characteristics like N-gram (Lavie et al., 1996; Berger et al., 1996; Nakajima and Yamamoto, 2001; Gupta et al., 2002). Some research efforts have achieved high performance in recall and pre- cision against correct splitting positions. Despite such a high performance, from the view point of translation, MT systems are not always able to translate the split sentences well. In order to supplement sentence splitting based on word-sequence characteristics, this paper intro- duces another measure of sentence similarity. In our splitting method, we generate candidates for splitting positions based on N-grams, and select the best combination of positions by measuring sentence similarity. This selection is based on the assumption that …

Content truncated to 500 words