Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4256 - 4266 July 10-15, 2022 ©2022 Association for Computational Linguistics Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation Siyu Lai1∗, Zhen Yang2 , Fandong Meng2, Xue Zhang 1, Yufeng Chen1†, Jinan Xu1 and Jie Zhou2 1Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing, China 2Pattern Recognition Center, WeChat AI, Tencent Inc, China {siyulai,xue_zhang,chenyf,jaxu}@bjtu.edu.cn, {zieenyang,fandongmeng,withtomzhou}@tencent.com Abstract Generating adversarial examples for Neu- ral Machine Translation (NMT) with single Round-Trip Translation (RTT) has achieved promising results by releasing the meaning- preserving restriction. However, a potential pitfall for this approach is that we cannot de- cide whether the generated examples are ad- versarial to the target NMT model or the aux- iliary backward one, as the reconstruction er- ror through the RTT can be related to ei- ther. To remedy this problem, we propose a new criterion for NMT adversarial examples based on the Doubly Round-Trip Translation (DRTT). Specifically, apart from the source- target-source RTT, we also consider the target- source-target one, which is utilized to pick out the authentic adversarial examples for the tar- get NMT model. Additionally, to enhance the robustness of the NMT model, we introduce the masked language models to construct bilin- gual adversarial pairs based on DRTT, which are used to train the NMT model directly. Extensive experiments on both the clean and noisy test sets (including the artificial and natu- ral noise) show that our approach substantially improves the robustness of NMT models. 1 Introduction In recent years, neural machine translation (NMT) (Cho et al., 2014; Bahdanau et al., 2014; Vaswani et al., 2017) has achieved rapid advancement in the translation performance (Yang et al., 2020; Lu et al., 2021). However, the NMT model is not al- ways stable enough, as its performance can drop significantly when small perturbations are added into the input sentences (Belinkov and Bisk, 2017; Cheng et al., 2020). Such perturbed inputs are often referred to as adversarial examples in the literature, and how to effectively generate and utilize adver- sarial examples for NMT is still an open question. ∗ Work was done when Siyu were interning at Pattern Recognition Center, WeChat AI, Tencent Inc, China. †Yufeng Chen is the corresponding author. 𝐱:神盾舰是一 艘巨大的战船 𝐱!:神盾舰是一 艘轻便的战船 𝐲′:Aegis is a huge warship 𝐲! " : Aegis is a light warship $𝐱:神盾舰是一 艘巨大的战船 $𝐱!:神盾舰是一 艘巨大的战船 source target source S2T T2S BLEU(𝐱, $𝐱) — BLEU(𝐱!, $𝐱!) Figure 1: An example of the source-target-source RTT process on a perturbed input xδ by replacing “巨大 (huge)” to “轻便 (light)”. Conventional approaches (Ebrahimi et al., 2018; Cheng et al., 2019) for generating NMT adversarial examples always follow the meaning-preserving assumption, i.e., an NMT adversarial example should preserve the meaning of the source sentence but destroy the translation performance drastically (Michel et al., 2019; Niu et al., 2020). With the meaning-preserving restriction, the researchers try to add perturbations on the source inputs as …

Content truncated to 500 words