Structural Supervision for Word Alignment and Machine Translation

Findings of the Association for Computational Linguistics: ACL 2022, pages 4084 - 4094 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Structural Supervision for Word Alignment and Machine Translation Lei Li1,2,3, Kai Fan∗ 2, Hongjia Li1,3, Chun Yuan3 1Department of Computer Science and Technology, Tsinghua University, Beijing, China 2Alibaba DAMO Academy, Alibaba Group Inc. 3Tsinghua Shenzhen International Graduate School, Peng Cheng Lab Abstract Syntactic structure has long been argued to be potentially useful for enforcing accurate word alignment and improving generalization perfor- mance of machine translation. Unfortunately, existing wisdom demonstrates its significance by considering only the syntactic structure of source tokens, neglecting the rich structural in- formation from target tokens and the structural similarity between the source and target sen- tences. In this work, we propose to incorporate the syntactic structure of both source and tar- get tokens into the encoder-decoder framework, tightly correlating the internal logic of word alignment and machine translation for multi- task learning. Particularly, we won’t lever- age any annotated syntactic graph of the tar- get side during training, so we introduce Dy- namic Graph Convolution Networks (DGCN) on observed target tokens to sequentially and simultaneously generate the target tokens and the corresponding syntactic graphs, and further guide the word alignment. On this basis, Hierar- chical Graph Random Walks (HGRW) are per- formed on the syntactic graphs of both source and target sides, for incorporating structured constraints on machine translation outputs. Ex- periments on four publicly available language pairs verify that our method is highly effective in capturing syntactic structure in different lan- guages, consistently outperforming baselines in alignment accuracy and demonstrating promis- ing results in translation quality. 1 Introduction Word alignment (Brown et al., 1993) aims to find the correspondence between tokens in a sentence pair. Neural machine translation (NMT) (Bahdanau et al., 2015; Vaswani et al., 2017) works by taking an end-to-end approach to incrementally predict the target translation from a source sentence, where no explicit word alignment is required during model training or decoding. Recently, there has been an ∗Corresponding author English French German Chinese 1-hop 2-hop Syntax-graph you re naive nsubj acomp conj cc Tu es naïve Du bist naiv 你 很 天真 nsubj advmod sb oa Regardless of direction and type: (1) the dependencies between 'you' and 're' and between 're' and 'naive' in English match the dependencies between ‘Du' and ‘bist' and between ‘bist' and ‘naiv' in German. (2) For English-French (Chinese) pairs, although there is no explicit dependency between 'Tu'(你) and 'es'(很), we can capture the implicit dependency by tracing the dependencies between 'Tu'( 你 ) and 'naïve'(天真) and between 'naïve'(天真) and 'es'(很). Figure 1: Similar dependencies of different languages. increasing interest (Zenkel et al., 2020; Chen et al., 2020, 2021; Zhang and van Genabith, 2021) in combining the two tasks through inducing accurate word alignment in neural translation models for improving translation quality. Intuitively, word alignment is helpful to enforce the domain-specific terminology or improve the translations of low-frequency tokens (Song et al., 2019; Dinu et al., 2019). …

Content truncated to 500 words