Improving Translation Model by Monolingual Data

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 330–336, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics Improving Translation Model by Monolingual Data∗ Ondˇrej Bojar and Aleˇs Tamchyna bojar@ufal.mff.cuni.cz, a.tamchyna@gmail.com Institute of Formal and Applied Linguistics, Faculty of Mathematics and Physics, Charles University in Prague Abstract We use target-side monolingual data to ex- tend the vocabulary of the translation model in statistical machine translation. This method called “reverse self-training” improves the de- coder’s ability to produce grammatically cor- rect translations into languages with morphol- ogy richer than the source language esp. in small-data setting. We empirically evalu- ate the gains for several pairs of European languages and discuss some approaches of the underlying back-off techniques needed to translate unseen forms of known words. We also provide a description of the systems we submitted to WMT11 Shared Task. 1 Introduction Like any other statistical NLP task, SMT relies on sizable language data for training. However the par- allel data required for MT are a very scarce resource, making it difficult to train MT systems of decent quality. On the other hand, it is usually possible to obtain large amounts of monolingual data. In this paper, we attempt to make use of the monolingual data to reduce the sparseness of surface forms, an issue typical for morphologically rich lan- guages. When MT systems translate into such lan- guages, the limited size of parallel data often causes the situation where the output should include a word form never observed in the training data. Even though the parallel data do contain the desired word ∗ This work has been supported by the grants EuroMatrix- Plus (FP7-ICT-2007-3-231720 of the EU and 7E09003 of the Czech Republic), P406/10/P259, and MSM 0021620838. in other forms, a standard phrase-based decoder has no way of using it to generate the correct translation. Reverse self-training addresses this problem by incorporating the available monolingual data in the translation model. This paper builds upon the idea outlined in Bojar and Tamchyna (2011), describing how this technique was incorporated in the WMT Shared Task and extending the experimental evalu- ation of reverse self-training in several directions – the examined language pairs (Section 4.2), data size (Section 4.3) and back-off techniques (Section 4.4). 2 Related Work The idea of using monolingual data for improving the translation model has been explored in several previous works. Bertoldi and Federico (2009) used monolingual data for adapting existing translation models to translation of data from different domains. In their experiments, the most effective approach was to train a new translation model from “fake” parallel data consisting of target-side monolingual data and their machine translation into the source language by a baseline system. Ueffing et al. (2007) used a boot-strapping tech- nique to extend translation models using mono- lingual data. They gradually translated additional source-side sentences and selectively incorporated them and their translations in the model. Our technique also bears a similarity to de Gis- pert et al. (2005), in that we try to …

Content truncated to 500 words