Latent Domain Word Alignment for Heterogeneous Corpora

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 398–408, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Latent Domain Word Alignment for Heterogeneous Corpora Hoang Cuong and Khalil Sima’an Institute for Logic, Language and Computation University of Amsterdam Science Park 107, 1098 XG Amsterdam, The Netherlands {c.hoang,k.simaan}@uva.nl Abstract This work focuses on the insensitivity of ex- isting word alignment models to domain dif- ferences, which often yields suboptimal re- sults on large heterogeneous data. A novel latent domain word alignment model is pro- posed, which induces domain-conditioned lexical and alignment statistics. We propose to train the model on a heterogeneous corpus under partial supervision, using a small num- ber of seed samples from different domains. The seed samples allow estimating sharper, domain-conditioned word alignment statistics for sentence pairs. Our experiments show that the derived domain-conditioned statistics, once combined together, produce notable im- provements both in word alignment accuracy and in translation accuracy of their resulting SMT systems. 1 Introduction Word alignment currently constitutes the basis for phrase extraction and reordering in phrase-based systems, and its statistics provide lexical parame- ters used for smoothing the phrase pair estimates. For over two decades since IBM models (Brown et al., 1993) and the HMM alignment model (Vo- gel et al., 1996), word alignment remains an active research line, e.g., see recent work (Simion et al., 2013; Tamura et al., 2014; Chang et al., 2014). During the past years we witnessed an increas- ing need to collect and use large heterogeneous par- allel corpora from different domains and sources, e.g., News, Wikipedia, Parliament Proceedings. It is tacitly assumed that assembling a larger corpus should improve a phrase-based system coverage and performance. Recent work (Sennrich et al., 2013; Carpuat et al., 2014; Cuong and Sima’an, 2014b; Kirchhoff and Bilmes, 2014; Cuong and Sima’an, 2014a) shows that this is not necessarily true as phrase translations as well as (bi- and monolingual) word co-occurrence statistics could differ across do- mains. This suggests that the word alignment qual- ity obtained from IBM and HMM alignment models might also be affected in heterogeneous corpora. Intuitively, in heterogeneous data certain words are present across many domains, whereas oth- ers are more specific to few domains. This sug- gests that the translation probabilities for words will be as fractioned as the diversity of its translations across the domains. Furthermore, because the IBM and HMM alignment models use context-insensitive conditional probabilities, in heterogeneous corpora the estimates of these probabilities will be aggre- gated over different domains. Both issues could lead to suboptimal word alignment quality. Surprisingly, the insensitivity of the existing IBM and HMM alignment models to domain differences has not received much attention thus far (see the study of Bach et al. (2008) and Gao et al. (2011) for reference in the literature). We conjecture that this is because it is not fully clear how to define what con- stitutes a (sub)-domain. In this paper we propose to …

Content truncated to 500 words