Multi-Task Word Alignment Triangulation for Low-Resource Languages

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1221–1226, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Multi-Task Word Alignment Triangulation for Low-Resource Languages Tomer Levinboim and David Chiang Department of Computer Science and Engineering University of Notre Dame {levinboim.1,dchiang}@nd.edu Abstract We present a multi-task learning approach that jointly trains three word alignment mod- els over disjoint bitexts of three languages: source, target and pivot. Our approach builds upon model triangulation, following Wang et al., which approximates a source-target model by combining source-pivot and pivot-target models. We develop a MAP-EM algorithm that uses triangulation as a prior, and show how to extend it to a multi-task setting. On a low-resource Czech-English corpus, using French as the pivot, our multi-task learning ap- proach more than doubles the gains in both F- and Bleuscores compared to the interpolation approach of Wang et al. Further experiments reveal that the choice of pivot language does not significantly affect performance. 1 Introduction Word alignment (Brown et al., 1993; Vogel et al., 1996) is a fundamental task in the machine transla- tion (MT) pipeline. To train good word alignment models, we require access to a large parallel corpus. However, collection of parallel corpora has mostly focused on a small number of widely-spoken lan- guages. As such, resources for almost any other pair are either limited or non-existent. To improve word alignment and MT in a low- resource setting, we design a multitask learning approach that utilizes parallel data of a third lan- guage, called the pivot language (§3). Specifi- cally, we derive an efficient and easy-to-implement MAP-EM-like algorithm that jointly trains source- target, source-pivot and pivot-target alignment mod- els, each on its own bitext, such that each model ben- efits from observations made by the other two. Our method subsumes the model interpolation ap- proach of Wang et al. (2006), who independently train these three models and then interpolate the source-target model with an approximate source- target model, constructed by combining the source- pivot and pivot-target models. Pretending that Czech-English is low-resource, we conduct word alignment and MT experi- ments (§4). With French as the pivot, our approach significantly outperforms the interpolation method of Wang et al. (2006) on both alignment F- and Bleu scores. Somewhat surprisingly, we find that our ap- proach is insensitive to the choice of pivot language. 2 Triangulation and Interpolation Wang et al. (2006) focus on learning a word align- ment model without a source-target corpus. To do so, they assume access to both source-pivot and pivot-target bitexts on which they independently train a source-pivot word alignment model Θsp and a pivot-target model Θpt. They then combine the two models by marginalizing over the pivot language, re- sulting in an approximate source-target model Θ� st. This combination process is referred to as triangu- lation (see §5). In particular, they construct the triangulated source-target t-table t� st from the source-pivot and pivot-target t-tables tsp, tpt using the following ap- …

Content truncated to 500 words