Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3867–3883 August 1–6, 2021. ©2021 Association for Computational Linguistics 3867 Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation Prakhar Gupta♣ Yulia Tsvetkov♠ Jeffrey P. Bigham♣,♥ ♣Language Technologies Institute, Carnegie Mellon University ♠Paul G. Allen School of Computer Science & Engineering, University of Washington ♥Human-Computer Interaction Institute, Carnegie Mellon University prakharg@cs.cmu.edu, yuliats@cs.washington.edu, jbigham@cs.cmu.edu Abstract Open-domain neural dialogue models have achieved high performance in response rank- ing and evaluation tasks. These tasks are formulated as a binary classification of re- sponses given in a dialogue context, and mod- els generally learn to make predictions based on context-response content similarity. How- ever, over-reliance on content similarity makes the models less sensitive to the presence of in- consistencies, incorrect time expressions and other factors important for response appro- priateness and coherence. We propose ap- proaches for automatically creating adversar- ial negative training data to help ranking and evaluation models learn features beyond con- tent similarity. We propose mask-and-fill and keyword-guided approaches that generate neg- ative examples for training more robust dia- logue systems. These generated adversarial re- sponses have high content similarity with the contexts but are either incoherent, inappropri- ate or not fluent. Our approaches are fully data-driven and can be easily incorporated in existing models and datasets. Experiments on classification, ranking and evaluation tasks across multiple datasets demonstrate that our approaches outperform strong baselines in pro- viding informative negative examples for train- ing dialogue systems.1 1 Introduction Due to growing availability of dialogue corpora (Li et al., 2017; Zhang et al., 2018; Smith et al., 2020) and the advancement of neural architectures (Rad- ford et al., 2019; Brown et al., 2020; Devlin et al., 2019), dialogue systems have achieved consider- able success. As typically formulated, dialogue models generate one or more candidate responses 1Code and data are publicly available //github.com/prakharguptaz/Adv_gen_https: dialogue to a provided context, consisting of past dialogue turns. Dialogue ranking (Zhou et al., 2018; Wu et al., 2019) and evaluation models (Tao et al., 2018; Yi et al., 2019; Sato et al., 2020), in turn, are deployed to select and score candidate responses according to coherence and appropriateness. Ranking and evaluation models are generally trained using true positive responses and randomly selected negative responses, which raises two is- sues. First, random negative candidates often have low content similarity with the context, and thus models learn to associate response coherence and appropriateness with content similarity (Yuan et al., 2019; Whang et al., 2021; Sai et al., 2020). In real systems, generated response candidates tend to be more similar in terms of content, and so other factors (e.g., time expressions, dialogue acts, in- consistencies) tend to be more important. Second, randomly selecting candidates as negative exam- ples in an open domain context can result in false negatives, leading to misclassification of appropri- ate responses. To make dialogue models more robust to the spurious pattern of content similarity, prior work proposed to leverage adversarial and counterfactual examples (Kaushik et al., 2020; …

Content truncated to 500 words