Factored Machine Translation Systems for Russian-English

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 154–157, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics Factored Machine Translation Systems for Russian-English St´ephane Huet, Elena Manishina and Fabrice Lef`evre Universit´e d’Avignon, LIA/CERI, France FirstName.LastName@univ-avignon.fr Abstract We describe the LIA machine transla- tion systems for the Russian-English and English-Russian translation tasks. Various factored translation systems were built us- ing MOSES to take into account the mor- phological complexity of Russian and we experimented with the romanization of un- translated Russian words. 1 Introduction This paper presents the factored phrase-based Machine Translation (MT) systems (Koehn and Hoang, 2007) developed at LIA, for the Russian- English and English-Russian translation tasks at WMT’13. These systems use only data provided for the evaluation campaign along with the LDC English Gigaword corpus. We summarize in Section 2 the resources used and the main characteristics of the systems based on the MOSES toolkit (Koehn et al., 2007). Sec- tion 3 reports experiments on the use of fac- tored translation models. Section 4 describes the transliteration process used to improve the Russian to English task. Finally, we conclude in Section 5. 2 System Architecture 2.1 Pre-processing The corpora available for the workshop were pre- processed using an in-house script that normal- izes quotes, dashes, spaces and ligatures. Long sentences or sentences with many numeric or non-alphanumeric characters were also discarded. Since the Yandex corpus is provided as lower- cased, we decided to lowercase all the other cor- pora. The same pipeline was applied to the LDC Gigaword; also only the documents classified as “story” were retained. Table 1 summarizes the used data and introduces designations that we fol- low in the remainder of this paper to refer to these corpora. Russian is a morphologically rich language with nouns, adjectives and verbs inflected for case, number and gender. This property requires in- troducing morphological information inside the MT system to handle the lack of many inflec- tional forms inside training corpora. For this purpose, each corpus was previously tagged with Part-of-Speech (PoS) tags. The tagger TREE- TAGGER (Schmid, 1995) was selected for its good performance on several comparable tasks. The Russian tagger associates each word (e.g. ÿùèêà (boxes)) with a complex PoS including morphological information (e.g. “Ncmpnn” for “Noun Type=common Gender=masculine Num- ber=plural Case=nominative Animate=no”) and its lemma (e.g. ÿùèê (box)). A description of the Russian tagset can be found in (Sharoff et al., 2008). The English tagger provides also a lemma- tization and outputs PoS from the Penn Treebank tagset (Marcus et al., 1993) (e.g. “NNS” for “Noun plural”). In order to simplify the comparison of differ- ent setups, we used the tokenizer included in the TREETAGGER tool to process all the corpora. 2.2 Language Models Kneser-Ney discounted LMs were built from monolingual corpora using the SRILM toolkit (Stolcke, 2002). 5-gram LMs were trained for words, 7-gram LMs for lemmas and PoS. A LM was built separately on each monolingual cor- pus: mono-news-c and news-s. Since ldc was too large to be processed …

Content truncated to 500 words