Influence of Parser Choice on Dependency-Based MT

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 433–439, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics Influence of Parser Choice on Dependency-Based MT Martin Popel, David Mareˇcek, Nathan Green and Zdenˇek ˇZabokrtsk´y Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics {popel,marecek,green,zabokrtsky}@ufal.mff.cuni.cz Abstract Accuracy of dependency parsers is one of the key factors limiting the quality of dependency- based machine translation. This paper deals with the influence of various dependency pars- ing approaches (and also different training data size) on the overall performance of an English-to-Czech dependency-based statisti- cal translation system implemented in the Treex framework. We also study the relation- ship between parsing accuracy in terms of un- labeled attachment score and machine transla- tion quality in terms of BLEU. 1 Introduction In the last years, statistical n-gram models domi- nated the field of Machine Translation (MT). How- ever, their results are still far from perfect. Therefore we believe it makes sense to investigate alternative statistical approaches. This paper is focused on an analysis-transfer-synthesis translation system called TectoMT whose transfer representation has a shape of a deep-syntactic dependency tree. The system has been introduced by ˇZabokrtsk´y et al. (2008). The translation direction under consideration is English- to-Czech. It has been shown by Popel (2009) that the current accuracy of the dependency parser employed in this translation system is one of the limiting factors from the viewpoint of its output quality. In other words, the parsing phase is responsible for a large portion of translation errors. The biggest source of trans- lation errors in the referred study was (and prob- ably still is) the transfer phase, however the pro- portion has changed since and the relative impor- tance of the parsing phase has grown, because the tranfer phase errors have already been addressed by improvements based on Hidden Markov Tree Mod- els for lexical and syntactic choice as shown by ˇZabokrtsk´y and Popel (2009), and by context sensi- tive translation models based on maximum entropy as described by Mareˇcek et al. (2010). Our study proceeds along two directions. First, we train two state-of-the-art dependency parsers on training sets with varying size. Second, we use five parsers based on different parsing techniques. In both cases we document the relation between parsing accuracy (in terms of Unlabeled Attachment Score, UAS) and translation quality (estimated by the well known BLEU metric). The motivation behind the first set of experiments is that we can extrapolate the learning curve and try to predict how new advances in dependency parsing can affect MT quality in the future. The second experiment series is motivated by the hypothesis that parsers based on different ap- proaches are likely to have a different distribution of errors, even if they can have competitive perfor- mance in parsing accuracy. In dependency parsing metrics, all types of incorrect edges typically have the same weight,1 but some incorrect edges can be more harmful than others from the MT viewpoint. For instance, an …

Content truncated to 500 words