Enhancing Neural Models with Vulnerability via Adversarial Attack

Proceedings of the 28th International Conference on Computational Linguistics, pages 1133–1146 Barcelona, Spain (Online), December 8-13, 2020 1133 Enhancing Neural Models with Vulnerability via Adversarial Attack Rong Zhang1,∗, Qifei Zhou1,∗, Bo An2, Weiping Li1,†, Tong Mo1, Bo Wu3 1Peking University, Beijing, China 2Institute of Software, Chinese Academy of Sciences, Beijing, China 3MIT-IBM Watson AI Lab, Massachusetts, United States {rzhangpku, qifeizhou}@pku.edu.cn, anbo@iscas.ac.cn {wpli, motong}@ss.pku.edu.cn, bo.wu@ibm.com Abstract Natural Language Sentence Matching (NLSM) serves as the core of many natural language pro- cessing tasks. 1) Most previous work develops a single specific neural model for NLSM tasks. 2) There is no previous work considering adversarial attack to improve the performance of NLSM tasks. 3) Adversarial attack is usually used to generate adversarial samples that can fool neu- ral models. In this paper, we first find a phenomenon that different categories of samples have different vulnerabilities. Vulnerability is the difficulty degree in changing the label of a sample. Considering the phenomenon, we propose a general two-stage training framework to enhance neural models with Vulnerability via Adversarial Attack (VAA). We design criteria to mea- sure the vulnerability which is obtained by adversarial attack. VAA framework can be adapted to various neural models by incorporating the vulnerability. In addition, we prove a theorem and four corollaries to explain the factors influencing vulnerability effectiveness. Experimen- tal results show that VAA significantly improves the performance of neural models on NLSM datasets. The results are also consistent with the theorem and corollaries. The code is released on https://github.com/rzhangpku/VAA. 1 Introduction Natural Language Sentence Matching (NLSM) aims to compare two sentences and identify the relation- ship between them (Wang et al., 2017). NLSM serves as the core of many natural language processing tasks, such as question answering and information retrieval (Wang et al., 2016). 1) Most previous work develops a single specific neural model for NLSM tasks. We propose a general framework that can enhance various neural models proposed by previous work. 2) There is no previous work considering adversarial attack to improve the performance of NLSM tasks. This paper is the first to take advantage of adversarial attack to improve the performance of NLSM tasks. 3) Adversarial attack is usually used to generate adversarial samples that can fool neural models. In contrast, we take advantage of adversarial attack to enhance neural models. We introduce a concept vulnerability, the difficulty degree in changing the label of a sample. Inspired by the finding that adversarial samples are more vulnerable than normal ones (Zhou et al., 2020), we first find a phenomenon that different categories of samples have different vulnerabilities. That is, it is easy to change the labels of samples belonging to some categories, whereas it is difficult for others. Here we take Quora Question Pairs (QQP) dataset as an example. If we want a paraphrase pair to be non-paraphrase, changing one word is sufficient. For example, simply adding “not” or “hardly” can completely change the meaning of a sentence. However, it is difficult to make two irrelevant sentences have the …

Content truncated to 500 words