PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation

Proceedings of the 29th International Conference on Computational Linguistics, pages 5085–5097 October 12–17, 2022. 5085 PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation Juncheng Wan1∗, Jian Yang2*, Shuming Ma3, Dongdong Zhang3, Weinan Zhang1 †, Yong Yu1 , Zhoujun Li2 1Shanghai Jiao Tong University 2State Key Lab of Software Development Environment, Beihang University 3Microsoft Research Asia {junchengwan,wnzhang,yyu}@apex.sjtu.edu.cn; {jiaya, lizj}@buaa.edu.cn; {shumma, dozhang}@microsoft.com Abstract While end-to-end neural machine translation (NMT) has achieved impressive progress, noisy input usually leads models to become fragile and unstable. Generating adversarial exam- ples as the augmented data has been proved to be useful to alleviate this problem. Exist- ing methods for adversarial example generation (AEG) are word-level or character-level, which ignore the ubiquitous phrase structure. In this paper, we propose a Phrase-level Adversarial Example Generation (PAEG) framework to en- hance the robustness of the translation model. Our method further improves the gradient- based word-level AEG method by adopting a phrase-level substitution strategy. We ver- ify our method on three benchmarks, includ- ing LDC Chinese-English, IWSLT14 German- English, and WMT14 English-German tasks. Experimental results demonstrate that our ap- proach significantly improves translation per- formance and robustness to noise compared to previous strong baselines. 1 Introduction Recently, neural machine translation (NMT) has effectively improved translation quality. NMT has shown state-of-the-art performance for many language pairs (Wu et al., 2016; Hassan et al., 2018; Vaswani et al., 2017). Various architec- tures (Sutskever et al., 2014; Bahdanau et al., 2015; Gehring et al., 2017; Vaswani et al., 2017) bring many appealing properties. Most NMT systems heavily rely on high-quality parallel data and per- form poorly in noisy input. With the noise rising in the source sentence, NMT tends to be more vul- nerable (Szegedy et al., 2014; Goodfellow et al., 2015), due to the output prediction of the decoder easily intervened by the other words (Cheng et al., 2018). A slight disturbance like a random permu- tation can damage the translation quality dramati- ∗Equal Contribution. †Corresponding author. Original Sentence A cooked hot dog in a bun with ketchup and relish. Word-level AEG A cooked warm dog in a bun with ketchup and relish. Phrase-level AEG A cooked sausage rolls in a bun with ketchup and relish. Table 1: An example of adversarial example generation (AEG). When the word “hot” is selected, word-level adversarial example generation method substitutes “hot” to “warm”. The phrase-level method substitutes the whole phrase “hot dog” to “sausage rolls”. cally (Belinkov and Bisk, 2018). Even replacing a word with a synonym in the source input, the NMT model can be cheated and the target output can not be translated correctly. To improve the robustness of the NMT model, previous works propose to construct the adversar- ial examples by manipulating hidden features or discrete text input. These adversarial examples are used as augmented data for the training of the NMT model. To attack hidden features, Cheng et al. (2018) added perturbations in the input at the feature level for adversarial stability training. To generate discrete adversarial input, Ebrahimi et …

Content truncated to 500 words