Improving Word Alignment using Word Similarity

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1840–1845, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Improving Word Alignment using Word Similarity Theerawat Songyot Dept of Computer Science University of Southern California songyot@usc.edu David Chiang† Dept of Computer Science and Engineering University of Notre Dame dchiang@nd.edu Abstract We show that semantic relationships can be used to improve word alignment, in ad- dition to the lexical and syntactic features that are typically used. In this paper, we present a method based on a neural net- work to automatically derive word simi- larity from monolingual data. We present an extension to word alignment models that exploits word similarity. Our exper- iments, in both large-scale and resource- limited settings, show improvements in word alignment tasks as well as translation tasks. 1 Introduction Word alignment is an essential step for learn- ing translation rules in statistical machine trans- lation. The task is to find word-level transla- tion correspondences in parallel text. Formally, given a source sentence e consisting of words e1, e2, . . . , el and a target sentence f consisting of words f1, f2, . . . , fm, we want to infer an alignment a, a sequence of indices a1, a2, . . . , am which indicates, for each target word fi, the corre- sponding source word eai or a null word. Machine translation systems, including state-of-the-art sys- tems, then use the word-aligned corpus to extract translation rules. The most widely used methods, the IBM mod- els (Brown et al., 1993) and HMM (Vogel et al., 1996), define a probability distribution p(f, a | e) that models how each target word fi is gener- ated from a source word eai with respect to an alignment a. The models, however, tend to mis- align low-frequency words as they have insuffi- cient training samples. The problem can get worse in low-resource languages. Two branches of re- search have tried to alleviate the problem. The †Most of the work reported here was performed while the second author was at the University of Southern California. first branch relies solely on the parallel data; how- ever, additional assumptions about the data are re- quired. This includes, but is not limited to, ap- plying prior distributions (Mermer and Sarac¸lar, 2011; Vaswani et al., 2012) or smoothing tech- niques (Zhang and Chiang, 2014). The other branch uses information learned from monolin- gual data, which is generally easier to acquire than parallel data. Previous work in this branch mostly involves applying syntactic constraints (Yamada and Knight, 2001; Cherry and Lin, 2006; Wang and Zong, 2013) and syntactic features (Toutanova et al., 2002) into the models. The use of syntac- tic relationships can, however, be limited between historically unrelated language pairs. Our motivation lies in the fact that a meaningful sentence is not merely a grammatically structured sentence; its semantics can provide insightful in- formation for the task. For example, suppose that the models are uncertain about aligning e …

Content truncated to 500 words