Graph Neural Networks for Multiparallel Word Alignment

Findings of the Association for Computational Linguistics: ACL 2022, pages 1384 - 1396 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Graph Neural Networks for Multiparallel Word Alignment Ayyoob Imani1, Lütfi Kerem ¸Senel1, Masoud Jalili Sabet1, François Yvon2, Hinrich Schütze1 1Center for Information and Language Processing (CIS), LMU Munich, Germany 2Université Paris-Saclay, CNRS, LISN, France {ayyoob, masoud, lksenel}@cis.lmu.de, francois.yvon@limsi.fr Abstract After a period of decrease, interest in word alignments is increasing again for their use- fulness in domains such as typological re- search, cross-lingual annotation projection and machine translation. Generally, alignment al- gorithms only use bitext and do not make use of the fact that many parallel corpora are multiparallel. Here, we compute high-quality word alignments between multiple language pairs by considering all language pairs to- gether. First, we create a multiparallel word alignment graph, joining all bilingual word alignment pairs in one graph. Next, we use graph neural networks (GNNs) to exploit the graph structure. Our GNN approach (i) uti- lizes information about the meaning, position and language of the input words, (ii) incorpo- rates information from multiple parallel sen- tences, (iii) adds and removes edges from the initial alignments, and (iv) yields a prediction model that can generalize beyond the training sentences. We show that community detec- tion provides valuable information for multi- parallel word alignment. Our method outper- forms previous work on three word alignment datasets and on a downstream task. 1 Introduction Word alignments are crucial for statistical machine translation (Koehn et al., 2003) and useful for many other multilingual tasks such as neural machine translation (Alkhouli and Ney, 2017; Alkhouli et al., 2016), typological analysis (Lewis and Xia, 2008; Östling, 2015; Asgari and Schütze, 2017) and annotation projection (Yarowsky and Ngai, 2001; Fossum and Abney, 2005; Wisniewski et al., 2014; Huck et al., 2019). The rise of deep learning initially led to a temporary plateau, but interest in word alignments is now increasing, demonstrated by several recent publications (Jalili Sabet et al., 2020; Chen et al., 2020; Dou and Neubig, 2021). While word alignment is usually considered for bilingual corpora, our work addresses the problem eat thistles thorns produce the the of will you you will for and It . fields plants and , Figure 1: Alignment graph for the verse “It will pro- duce thorns and thistles for you, and you will eat the plants of the field.” in a 12-way multiparallel corpus. Colors represent languages. Each English (yellow) node is annotated with its word. Red dashed lines cut links that incorrectly connect distinct concepts. We ex- ploit community detection algorithms to detect distinct concepts. This provides valuable information for our GNN model and improves word alignments. of word alignment in multiparallel corpora, con- taining sentence level parallel text in more than two languages. Examples of multiparallel corpora are JW300 (Agi´c and Vuli´c, 2019), PBC (Mayer and Cysouw, 2014) which covers the highest number of languages (1334), and Tatoeba.1 While the per- language amount of data provided in such corpora is less than bilingual …

Content truncated to 500 words