Translation Quality Indicators for Pivot-based Statistical MT

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 811–818, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Translation Quality Indicators for Pivot-based Statistical MT Michael Paul and Eiichiro Sumita National Institute of Information and Communications Technology MASTAR Project Kyoto, Japan michael.paul@nict.go.jp Abstract Recent research on multilingual statisti- cal machine translation focuses on the us- age of pivot languages in order to over- come resource limitations for certain lan- guage pairs. This paper provides new in- sights into what factors make a good pivot language and investigates the impact of these factors on the overall pivot transla- tion performance. Pivot-based SMT ex- periments translating between 22 Indo- European and Asian languages were used to analyze the impact of eight factors (lan- guage family, vocabulary, sentence length, language perplexity, translation model en- tropy, reordering, monotonicity, engine performance) on pivot translation perfor- mance. The results showed that 81% of system performance variations can be ex- plained by these factors. 1 Introduction The translation quality of statistical machine trans- lation (SMT) approaches heavily depends on the amount and coverage of bilingual language re- sources available to train the statistical models. There exist several data collection initiatives1 amassing and distributing large amounts of tex- tual data. For frequently used language pairs like French-English, large text data sets are readily available. However, for less frequently used lan- guage pairs only a limited amount of bilingual re- sources are available, if any at all. In order to overcome language resource limita- tions, recent research on SMT focuses on the us- age of pivot languages (de Gispert and Marino, 2006; Utiyama and Isahara, 2007; Wu and Wang, 2007; Bertoldi et al., 2008). Instead of a direct translation between two languages where only a 1LDC: http://www.ldc.upenn.edu, ELRA: http://www.elra.info limited amount of bilingual resources is available, the pivot translation approach makes use of a third language that is more appropriate due to the avail- ability of more bilingual corpora and/or its relat- edness towards either the source or the target lan- guage. For most recent research efforts, English is the pivot language of choice due to the richness of available language resources. However, recent re- search on pivot translation has shown that the us- age of non-English pivot languages can improve translation quality for certain language pairs (Paul et al., 2009; Leusch et al., 2010). Concerning the contribution of aspects of differ- ent language pairs on the quality of machine trans- lation, (Birch et al., 2008) identified three features (morphological complexity, amount of reordering, historical relatedness) for predicting success of MT in translations between the official languages of the European Union. Moreover, (Koehn et al., 2009) investigated an additional feature (transla- tion model complexity) using the JRC-Aquis cor- pus covering not only Indo-European languages, but also one semitic and three Finno-Ugric lan- guages. This paper differs from previous research in the following aspects: we focus on the frame- work of pivot translation, where a target language translation of a source language input is obtained …

Content truncated to 500 words