Evaluating and Enhancing the Robustness of Dialogue Systems: A Case Study on a Negotiation Agent

Proceedings of NAACL-HLT 2019, pages 3325–3335 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3325 Evaluating and Enhancing the Robustness of Dialogue Systems: A Case Study on a Negotiation Agent Minhao Cheng University of California Los Angeles mhcheng@cs.ucla.edu Wei Wei Google Cloud AI wewei@google.com Cho-Jui Hsieh University of California Los Angeles chohsieh@cs.ucla.edu Abstract Recent research has demonstrated that goal- oriented dialogue agents trained on large datasets can achieve striking performance when interacting with human users. In real world applications, however, it is important to ensure that the agent performs smoothly in- teracting with not only regular users but also those malicious ones who would attack the system through interactions in order to achieve goals for their own advantage. In this paper, we develop algorithms to evaluate the robust- ness of a dialogue agent by carefully designed attacks using adversarial agents. Those attacks are performed in both black-box and white- box settings. Furthermore, we demonstrate that adversarial training using our attacks can significantly improve the robustness of a goal- oriented dialogue system. On a case-study of the negotiation agent developed by (Lewis et al., 2017), our attacks reduced the average ad- vantage of rewards between the attacker and the trained RL-based agent from 2.68 to −5.76 on a scale from −10 to 10 for randomized goals. Moreover, with the proposed adversar- ial training, we are able to improve the robust- ness of negotiation agents by 1.5 points on av- erage against all our attacks. 1 Introduction Crafting an intelligent agent to communicate in the dialogue system using natural languages has been a long-standing problem in AI. It requires designing an agent to understand, plan and gen- erate natural language to achieve different goals such as question-answering, cooperation, negotia- tion etc (Vinyals and Le, 2015; Li et al., 2017; Ser- ban et al., 2016; Dhingra et al., 2016; Serban et al., 2016). Inspired by recent successes in deep neural networks, (Lewis et al., 2017) has recently devel- oped an end-to-end learning framework to train a recurrent neural network (RNN)-based negotiation agent in goal-oriented dialogue systems. This NN- based technique has been identified as one of the state-of-the-arts and has been applied to several other tasks (Bahdanau et al., 2014; Luong et al., 2015; Rush et al., 2015; Chan et al., 2016). Although NN-based dialogue agents have shown convincing performance on several tasks, it is not clear whether they also work well when facing malicious users or agents. To answer this question, we study how to evaluate the robustness of a goal-oriented dialogue system. For simplicity, we consider a goal-oriented agent A that aims to maximize some score, and define the “robustness” of A as the worst-case performance under any fea- sible agent A′. We also call A′ an adversarial agent that tries to “attack” A since it aims to mini- mize A’s score. The problem of evaluating the ro- bustness of A can then be solved by designing an adversarial agent to attack A. For instance, con- …

Content truncated to 500 words