Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 486–496 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 486 Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models Tong Niu and Mohit Bansal UNC Chapel Hill {tongn, mbansal}@cs.unc.edu Abstract We present two categories of model-agnostic adversarial strategies that reveal the weak- nesses of several generative, task-oriented di- alogue models: Should-Not-Change strate- gies that evaluate over-sensitivity to small and semantics-preserving edits, as well as Should- Change strategies that test if a model is over- stable against subtle yet semantics-changing modifications. We next perform adversarial training with each strategy, employing a max- margin approach for negative generative ex- amples. This not only makes the target di- alogue model more robust to the adversarial inputs, but also helps it perform significantly better on the original inputs. Moreover, train- ing on all strategies combined achieves fur- ther improvements, achieving a new state-of- the-art performance on the original task (also verified via human evaluation). In addition to adversarial training, we also address the ro- bustness task at the model-level, by feeding it subword units as both inputs and outputs, and show that the resulting model is equally com- petitive, requires only 1/4 of the original vo- cabulary size, and is robust to one of the adver- sarial strategies (to which the original model is vulnerable) even without adversarial training. 1 Introduction Adversarial evaluation aims at filling in the gap between potential train/test distribution mismatch and revealing how models will perform under real- world inputs containing natural or malicious noise. Recently, there has been substantial work on ad- versarial attacks in computer vision and NLP. Un- like vision, where one can simply add in impercep- tible perturbations without changing an image’s meaning, carrying out such subtle changes in text is harder since text is discrete in nature (Jia and We publicly release all our code and data at https: //github.com/WolfNiu/AdversarialDialogue Liang, 2017). Thus, some previous works have ei- ther avoided modifying original source inputs and only resorted to inserting distractive sentences (Jia and Liang, 2017), or have restricted themselves to introducing spelling errors (Belinkov and Bisk, 2018) and adding non-functioning tokens (Sha- lyminov et al., 2017). Furthermore, there has been limited adversarial work on generative NLP tasks, e.g., dialogue generation (Henderson et al., 2017), which is especially important because it is a crucial component of real-world virtual assistants such as Alexa, Siri, and Google Home. It is also a challenging and worthwhile task to keep the out- put quality of a dialogue system stable, because a conversation usually involves multiple turns, and a small mistake in an early turn could cascade into bigger misunderstanding later on. Motivated by this, we present a comprehensive adversarial study on dialogue models – we not only simulate imperfect inputs in the real world, but also launch intentionally malicious attacks on the model in order to assess them on both over- sensitivity and over-stability. Unlike most previ- ous works that exclusively focus on …

Content truncated to 500 words