From n-gram-based to CRF-based Translation Models

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 542–553, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics From n-gram-based to CRF-based Translation Models Thomas Lavergne Josep Maria Crego LIMSI/CNRS BP 133 F-91 403 Orsay C´edex {lavergne,jmcrego}@limsi.fr Alexandre Allauzen Franc¸ois Yvon LIMSI/CNRS & Uni. Paris Sud BP 133 F-91 403 Orsay C´edex {allauzen,yvon}@limsi.fr Abstract A major weakness of extant statistical ma- chine translation (SMT) systems is their lack of a proper training procedure. Phrase extrac- tion and scoring processes rely on a chain of crude heuristics, a situation judged problem- atic by many. In this paper, we recast the ma- chine translation problem in the familiar terms of a sequence labeling task, thereby enabling the use of enriched feature sets and exact train- ing and inference procedures. The tractabil- ity of the whole enterprise is achieved through an efficient implementation of the conditional random fields (CRFs) model using a weighted finite-state transducers library. This approach is experimentally contrasted with several con- ventional phrase-based systems. 1 Introduction A weakness of existing phrase-based SMT systems, that has been repeatedly highlighted, is their lack of a proper training procedure. Attempts to de- sign probabilistic models of phrase-to-phrase align- ments (e.g. (Marcu and Wong, 2002)) have thus far failed to overcome the related combinatorial prob- lems (DeNero and Klein, 2008) and/or to yield im- proved training heuristics (DeNero et al., 2006). Phrase extraction and scoring thus rely on a chain of heuristics see (Koehn et al., 2003), which evolve phrase alignments from “symmetrized” word-to- word alignments obtained with IBM models (Brown et al., 1990) and the like (Liang et al., 2006b; Deng and Byrne, 2006; Ganchev et al., 2008). Phrase scoring is also mostly heuristic and relies on an op- timized interpolation of several simple frequency- based scores. Overall, the training procedure of translation models within conventional phrase-based (or hierarchical) systems is generally considered un- satisfactory and the design of better estimation pro- cedures remains an active research area (Wuebker et al., 2010). To overcome the NP-hard problems that derive from the need to consider all possible permutations of the source sentence, we make here a radical simplification and consider training the translation model given a fixed segmentation and reordering. This idea is not new, and is one of the grounding principle of n-gram-based approaches (Casacuberta and Vidal, 2004; Mari˜no et al., 2006) in SMT. The novelty here is that we will use this assumption to re- cast machine translation (MT) in the familiar terms of a sequence labeling task. This reformulation allows us to make use of the efficient training and inference tools that exists for such tasks, most notably linear CRFs (Lafferty et al., 2001; Sutton and McCallum, 2006). It also en- ables to easily integrate linguistically informed (de- scribing morphological or morpho-syntactical prop- erties of phrases) and/or contextual features into the translation model. In return, in addition to having a better trained model, we also expect (i) to make estimation less sensible to data sparsity issues and …

Content truncated to 500 words