Joint Prediction of Word Alignment with Alignment Types

Joint Prediction of Word Alignment with Alignment Types Anahita Mansouri Bigvand Te Bu Anoop Sarkar School of Computing Science Simon Fraser University Burnaby, BC, Canada {amansour,tbu,anoop}@cs.sfu.ca Abstract Current word alignment models do not dis- tinguish between different types of alignment links. In this paper, we provide a new proba- bilistic model for word alignment where word alignments are associated with linguistically motivated alignment types. We propose a novel task of joint prediction of word align- ment and alignment types and propose novel semi-supervised learning algorithms for this task. We also solve a sub-task of predicting the alignment type given an aligned word pair. In our experimental results, the generative mod- els we introduce to model alignment types significantly outperform the models without alignment types. 1 Introduction Word alignment is an essential component in a sta- tistical machine translation (SMT) system. Soft alignments, or attention, are also an important com- ponent in neural machine translation (NMT) sys- tems. The classic generative model approach to word alignment is based on IBM models 1-5 (Brown et al., 1993) and the HMM model (Vogel et al., 1996; Och and Ney, 2000a). These traditional mod- els use unsupervised algorithms to learn alignments, relying on a large amount of parallel training data without hand annotated alignments. Supervised al- gorithms for word alignment have become more widespread with the availability of manually anno- tated word-aligned data and have shown promising results (Taskar et al., 2005; Blunsom and Cohn, 2006; Moore et al., 2006; Liang et al., 2006). Man- ually word-aligned data are valuable resources for SMT research, but they are costly to create and are only available for a handful of language pairs. Semi- supervised methods for word alignment combine hand-annotated word alignment data with parallel data without explicit word alignments. Even small amounts of hand-annotated word alignment data has been shown to improve the alignment and translation quality (Callison-Burch et al., 2004). In this paper, we provide a novel semi-supervised word alignment model that adds alignment type information to word alignments. Unsupervised or semi-supervised probabilistic word alignment models do not play a central role in neural machine translation (NMT) (Bahdanau et al., 2015; Sutskever et al., 2014; Luong et al., 2015; Chung et al., 2016). However, attention models, which are crucial for high-quality NMT, have been augmented with ideas from statistical word align- ment (Luong et al., 2015; Cohn et al., 2016). Other than machine translation, word alignments are also important in the best performing models for NLP tasks. They play a central role in learning para- phrases in a source language by doing round-trips from source to target and back using word align- ments (Ganitkevitch et al., 2013). Aligments also form the basis for learning multi-lingual word em- beddings (Faruqui and Dyer, 2014; Lu et al., 2015) and in the projection of syntactic and semantic an- notations from one language to another (Hwa et al., 2005; McDonald et al., 2011). Therefore, there is still a prominent role for word alignment in NLP; research into improvements …

Content truncated to 500 words