The impact of parse quality on syntactically-informed statistical machine translation

Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 62–69, Sydney, July 2006. c⃝2006 Association for Computational Linguistics The impact of parse quality on syntactically-informed statistical machine translation Chris Quirk and Simon Corston-Oliver Microsoft Research One Microsoft Way Redmond, WA 98052 USA {chrisq,simonco}@microsoft.com Abstract We investigate the impact of parse quality on a syntactically-informed statistical ma- chine translation system applied to techni- cal text. We vary parse quality by vary- ing the amount of data used to train the parser. As the amount of data increases, parse quality improves, leading to im- provements in machine translation output and results that significantly outperform a state-of-the-art phrasal baseline. 1 Introduction The current study is a response to a question that proponents of syntactically-informed machine translation frequently encounter: How sensitive is a syntactically-informed machine translation sys- tem to the quality of the input syntactic analysis? It has been shown that phrasal machine translation systems are not affected by the quality of the in- put word alignments (Koehn et al., 2003). This finding has generally been cast in favorable terms: such systems are robust to poor quality word align- ment. A less favorable interpretation of these re- sults might be to conclude that phrasal statistical machine translation (SMT) systems do not stand to benefit from improvements in word alignment. In a similar vein, one might ask whether con- temporary syntactically-informed machine trans- lation systems would benefit from improvements in parse accuracy. One possibility is that cur- rent syntactically-informed SMT systems are de- riving only limited value from the syntactic anal- yses, and would therefore not benefit from im- proved analyses. Another possibility is that syn- tactic analysis does indeed contain valuable infor- mation that could be exploited by machine learn- ing techniques, but that current parsers are not of sufficient quality to be of use in SMT. With these questions and concerns, let us be- gin. Following some background discussion we describe a set of experiments intended to elucidate the impact of parse quality on SMT. 2 Background We trained statistical machine translation systems on technical text. In the following sections we provide background on the data used for training, the dependency parsing framework used to pro- duce treelets, the treelet translation framework and salient characteristics of the target languages. 2.1 Dependency parsing Dependency analysis is an alternative to con- stituency analysis (Tesni`ere, 1959; Melˇcuk, 1988). In a dependency analysis of syntax, words di- rectly modify other words, with no intervening non-lexical nodes. We use the terms child node and parent node to denote the tokens in a depen- dency relation. Each child has a single parent, with the lexical root of the sentence dependent on a syn- thetic ROOT node. We use the parsing approach described in (Corston-Oliver et al., 2006). The parser is trained on dependencies extracted from the English Penn Treebank version 3.0 (Marcus et al., 1993) by using the head-percolation rules of (Yamada and Matsumoto, 2003). Given a sentence x, the goal of the …

Content truncated to 500 words