An Unsupervised Model for Joint Phrase Alignment and Extraction

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 632–641, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics An Unsupervised Model for Joint Phrase Alignment and Extraction Graham Neubig1,2 Taro Watanabe2, Eiichiro Sumita2, Shinsuke Mori1, Tatsuya Kawahara1 1Graduate School of Informatics, Kyoto University Yoshida Honmachi, Sakyo-ku, Kyoto, Japan 2National Institute of Information and Communication Technology 3-5 Hikari-dai, Seika-cho, Soraku-gun, Kyoto, Japan Abstract We present an unsupervised model for joint phrase alignment and extraction using non- parametric Bayesian methods and inversion transduction grammars (ITGs). The key con- tribution is that phrases of many granulari- ties are included directly in the model through the use of a novel formulation that memorizes phrases generated not only by terminal, but also non-terminal symbols. This allows for a completely probabilistic model that is able to create a phrase table that achieves com- petitive accuracy on phrase-based machine translation tasks directly from unaligned sen- tence pairs. Experiments on several language pairs demonstrate that the proposed model matches the accuracy of traditional two-step word alignment/phrase extraction approach while reducing the phrase table to a fraction of the original size. 1 Introduction The training of translation models for phrase- based statistical machine translation (SMT) systems (Koehn et al., 2003) takes unaligned bilingual train- ing data as input, and outputs a scored table of phrase pairs. This phrase table is traditionally gen- erated by going through a pipeline of two steps, first generating word (or minimal phrase) alignments, then extracting a phrase table that is consistent with these alignments. However, as DeNero and Klein (2010) note, this two step approach results in word alignments that are not optimal for the final task of generating phrase tables that are used in translation. As a so- lution to this, they proposed a supervised discrimi- native model that performs joint word alignment and phrase extraction, and found that joint estimation of word alignments and extraction sets improves both word alignment accuracy and translation results. In this paper, we propose the first unsuper- vised approach to joint alignment and extraction of phrases at multiple granularities. This is achieved by constructing a generative model that includes phrases at many levels of granularity, from minimal phrases all the way up to full sentences. The model is similar to previously proposed phrase alignment models based on inversion transduction grammars (ITGs) (Cherry and Lin, 2007; Zhang et al., 2008; Blunsom et al., 2009), with one important change: ITG symbols and phrase pairs are generated in the opposite order. In traditional ITG models, the branches of a biparse tree are generated from a non- terminal distribution, and each leaf is generated by a word or phrase pair distribution. As a result, only minimal phrases are directly included in the model, while larger phrases must be generated by heuris- tic extraction methods. In the proposed model, at each branch in the tree, we first attempt to gener- ate a phrase pair from the phrase pair distribution, falling back to ITG-based divide and conquer …

Content truncated to 500 words