Enhanced Language Representation with Label Knowledge for Span Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 456–470 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 456 Automatically Exposing Problems with Neural Dialog Models Dian Yu University of California, Davis dianyu@ucdavis.edu Kenji Sagae University of California, Davis sagae@ucdavis.edu Abstract Neural dialog models are known to suffer from problems such as generating unsafe and incon- sistent responses. Even though these problems are crucial and prevalent, they are mostly man- ually identified by model designers through in- teractions. Recently, some research instructs crowdworkers to goad the bots into triggering such problems. However, humans leverage su- perficial clues such as hate speech, while leav- ing systematic problems undercover. In this paper, we propose two methods including re- inforcement learning to automatically trigger a dialog model into generating problematic re- sponses. We show the effect of our methods in exposing safety and contradiction issues with state-of-the-art dialog models. 1 Introduction Language models, including dialog models, greatly benefit from training on large amounts of data with the objective of mimicking human generated sen- tences (Radford et al., 2019; Brown et al., 2020; Zhang et al., 2019a; Adiwardana et al., 2020; Roller et al., 2021). However, even with carefully pre- processed training data from online sources, neural dialog models are prone to issues including generic utterances, repetition, contradiction, and lack of safety (Li et al., 2016a; Welleck et al., 2020; Li et al., 2020; Roller et al., 2021; Xu et al., 2021). Compared to modularized dialog systems which are designed to avoid these problems (Yu et al., 2019; Paranjape et al., 2020), fixing these issues with end-to-end neural models is more challenging, which may hinder real world use of trained mod- els (Wolf et al., 2017; Simonite, 2021). We argue that before solving these problems using simulated data from simplified scenarios, we need to be able to probe the models and expose the problems in a dynamic way. Even though crucial limitations of neural dia- log models are prevalent, they are mostly manu- ally identified and categorized through interactions between model designers and the dialog system during qualitative analysis (Roller et al., 2021). Re- cent work proposes asking annotators to converse with dialog models while goading the model into generating problematic responses in a black-box attack setting. Although the data collected in this way can improve the performance of both problem classifiers and model generation, human annotators mostly rely on straightforward and intuitive strate- gies to collect the dataset, which may only expose superficial problems. For instance, Xu et al. (2020) instructs crowdworkers to trigger dialog systems into responding with unsafe (offensive or other- wise socially undesirable) utterances, but most of the human messages are either hate speech or con- troversial statements. Similarly, Nie et al. (2021) asks Mechanical Turkers to manually write con- tradicting dialogs for both humans and bots, or to interact with chatbots, where a frequent strat- egy is to ask factual questions intentionally leading to contradiction (e.g. ask “do you speak Spanish” after the bot says …

Content truncated to 500 words