The Hiero Machine Translation System: Extensions, Evaluation, and Analysis

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 779–786, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics The Hiero Machine Translation System: Extensions, Evaluation, and Analysis David Chiang, Adam Lopez, Nitin Madnani, Christof Monz, Philip Resnik, Michael Subotin Institute for Advanced Computer Studies (UMIACS) University of Maryland, College Park, MD 20742, USA {dchiang,alopez,nmadnani,christof,resnik,msubotin}@umiacs.umd.edu Abstract Hierarchical organization is a well known prop- erty of language, and yet the notion of hierarchi- cal structure has been largely absent from the best performing machine translation systems in recent community-wide evaluations. In this paper, we dis- cuss a new hierarchical phrase-based statistical ma- chine translation system (Chiang, 2005), present- ing recent extensions to the original proposal, new evaluation results in a community-wide evaluation, and a novel technique for fine-grained comparative analysis of MT systems. 1 Introduction Hierarchical organization is a well known prop- erty of language, and yet the notion of hierarchi- cal structure has, for the last several years, been absent from the best performing machine transla- tion systems in community-wide evaluations. Statis- tical phrase-based models (e.g. (Och and Ney, 2004; Koehn et al., 2003; Marcu and Wong, 2002)) charac- terize a source sentence f as a flat partition of non- overlapping subsequences, or “phrases”, ¯f1 · · · ¯fJ, and the process of translation involves selecting tar- get phrases ¯ei corresponding to the ¯f j and modify- ing their sequential order. The need for some way to model aspects of syntactic behavior, such as the tendency of constituents to move together as a unit, is widely recognized—the role of syntactic units is well attested in recent systematic studies of trans- lation (Fox, 2002; Hwa et al., 2002; Koehn and Knight, 2003), and their absence in phrase-based models is quite evident when looking at MT system output. Nonetheless, attempts to incorporate richer linguistic features have generally met with little suc- cess (Och et al., 2004a). Chiang (2005) introduces Hiero, a hierarchical phrase-based model for statistical machine transla- tion. Hiero extends the standard, non-hierarchical notion of “phrases” to include nonterminal sym- bols, which permits it to capture both word-level and phrase-level reorderings within the same framework. The model has the formal structure of a synchronous CFG, but it does not make any commitment to a linguistically relevant analysis, and it does not re- quire syntactically annotated training data. Chiang (2005) reported significant performance improve- ments in Chinese-English translation as compared with Pharaoh, a state-of-the-art phrase-based system (Koehn, 2004). In Section 2, we review the essential elements of Hiero. In Section 3 we describe extensions to this system, including new features involving named entities and numbers and support for a fourfold scale-up in training set size. Section 4 presents new evaluation results for Chinese-English as well as Arabic-English translation, obtained in the context of the 2005 NIST MT Eval exercise. In Section 5, we introduce a novel technique for fine-grained com- parative analysis of MT systems, which we em- ploy in analyzing differences between Hiero’s and …

Content truncated to 500 words