Combining Evaluation Metrics via Loss Functions

Constraining the Phrase-Based, Joint Probability Statistical Translation Model Alexandra Birch Chris Callison-Burch Miles Osborne School of Informatics University of Edinburgh 2 Buccleuch Place Edinburgh, EH8 9LW, UK a.c.birch-mayne@sms.ed.ac.uk Abstract The Joint Probability Model proposed by Marcu and Wong (2002) provides a probabilistic framework for modeling phrase-based statistical machine transla- tion (SMT). The model’s usefulness is, however, limited by the computational complexity of estimating parameters at the phrase level. We present a method of constraining the search space of the Joint Probability Model based on statistically and linguistically motivated word align- ments. This method reduces the complex- ity and size of the Joint Model and allows it to display performance superior to the standard phrase-based models for small amounts of training material. 1 Introduction Machine translation is a hard problem because of the highly complex, irregular and diverse nature of nat- ural languages. It is impossible to accurately model all the linguistic rules that shape the translation pro- cess, and therefore a principled approach uses sta- tistical methods to make optimal decisions given in- complete data. The original IBM Models (Brown et al., 1993) learned only word-to-word alignment probabilities which made it computationally feasible to estimate model parameters from large amounts of training data. Phrase-based SMT models, such as the Align- ment Template Model (Och, 2003b), improve on word-based models because phrases provide local context which leads to better lexical choice and more reliable local reordering. However, most phrase- based models extract their phrase pairs from previ- ously word-aligned corpora using ad-hoc heuristics. These models perform no search for optimal phrasal alignments. Even though this is an efficient strategy, it is a departure from the rigourous statistical frame- work of the IBM Models. Marcu and Wong (2002) proposed a Joint Proba- bility Model which directly estimates phrase trans- lation probabilities from the corpus. This model neither relies on potentially sub-optimal word align- ments nor on heuristics for phrase extraction. In- stead, it searches the phrasal alignment space, si- multaneously learning translation lexicons for both words and phrases. The Joint Model has been shown to outperform standard models on restricted data sets such as the small data track for Chinese-English in the 2004 NIST MT Evaluation (Przybocki, 2004). However, considering all possible phrases and all their possible alignments vastly increases the com- putational complexity of the Joint Model when com- pared to its word-based counterpart. This results in prohibitively slow training and heavy use of mem- ory resources. The large size of the model means that only a very small proportion of the alignment space can be searched, and this reduces the chances of finding optimum parameters. Furthermore, the complexity of the Joint Model makes it impossible to scale up to the larger training corpora available to- day, preventing the model from being more widely adopted. We propose a method of constraining the search 10 Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, pages 10-18, Cambridge, August 2006. ©2006 The Association for Machine Translation in the …

Content truncated to 500 words