Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6562–6577 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6562 Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach Simiao Zuo†∗, Chen Liang†, Haoming Jiang2, Xiaodong Liu⋄, Pengcheng He⋄, Jianfeng Gao⋄, Weizhu Chen⋄ and Tuo Zhao† †Georgia Institute of Technology 2Amazon ⋄Microsoft {simiaozuo,cliang73}@gatech.edu, jhaoming@amazon.com {xiaodl,Pengcheng.H,jfgao,wzchen}@microsoft.com, tourzhao@gatech.edu Abstract Adversarial regularization has been shown to improve the generalization performance of deep learning models in various natural lan- guage processing tasks. Existing works usu- ally formulate the method as a zero-sum game, which is solved by alternating gradient de- scent/ascent algorithms. Such a formulation treats the adversarial and the defending play- ers equally, which is undesirable because only the defending player contributes to the gener- alization performance. To address this issue, we propose Stackelberg Adversarial Regular- ization (SALT), which formulates adversarial regularization as a Stackelberg game. This formulation induces a competition between a leader and a follower, where the follower gen- erates perturbations, and the leader trains the model subject to the perturbations. Different from conventional approaches, in SALT, the leader is in an advantageous position. When the leader moves, it recognizes the strategy of the follower and takes the anticipated fol- lower’s outcomes into consideration. Such a leader’s advantage enables us to improve the model fitting to the unperturbed data. The leader’s strategic information is captured by the Stackelberg gradient, which is obtained us- ing an unrolling algorithm. Our experimental results on a set of machine translation and nat- ural language understanding tasks show that SALT outperforms existing adversarial regu- larization baselines across all tasks. Our code is publicly available. 1 Introduction Adversarial regularization (Miyato et al., 2017) has been shown to improve the generalization perfor- mance of deep learning models in various natural language processing (NLP) tasks, such as language modeling (Wang et al., 2019b), machine translation (Sato et al., 2019), natural language understand- ing (Jiang et al., 2020), and reading comprehen- ∗Corresponding author. sion (Jia and Liang, 2017). However, even though significant progress has been made, the power of adversarial regularization is not fully harnessed. Conventional adversarial regularization is for- mulated as a zero-sum game (a min-max optimiza- tion problem), where two players seek to mini- mize/maximize their utility functions. In this for- mulation, an adversarial player composes perturba- tions, and a defending player solves for the model parameters subject to the perturbed inputs. Ex- isting algorithms find the equilibrium of this zero- sum game using alternating gradient descent/ascent (Madry et al., 2018). For example, in a classifica- tion problem, the adversarial player first generates the input perturbations by running projected gradi- ent ascent to maximize a loss function, and then the defending player updates the model using gradient descent, trying to decrease the classification error. Notice that in this case, neither of the players know the strategy of its competitor, i.e., the model does not know how the perturbations are generated, and vice versa. In other words, the two players are of the same priority, and …

Content truncated to 500 words