SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4489–4505 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4489 SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training Huda Khayrallah Johns Hopkins University huda@jhu.edu Jo˜ao Sedoc New York University jsedoc@stern.nyu.edu Abstract Non-task-oriented dialog models suffer from poor quality and non-diverse responses. To overcome limited conversational data, we ap- ply Simulated Multiple Reference Training (SMRT; Khayrallah et al., 2020), and use a paraphraser to simulate multiple responses per training prompt. We find SMRT improves over a strong Transformer baseline as measured by human and automatic quality scores and lex- ical diversity. We also find SMRT is compa- rable to pretraining in human evaluation qual- ity, and outperforms pretraining on automatic quality and lexical diversity, without requiring related-domain dialog data. 1 Introduction Non-task-oriented dialog is a low-resource NLP task. While large and noisy related corpora ex- ist (e.g. movie subtitles, social media, and ir- clogs; Serban et al., 2018), the publicly-released curated corpora are small. Serban et al. note that smaller corpora have lower lexical diversity and topic coverage, leading to models with poor quality non-diverse responses. Pretraining on larger data may improve performance, but requires a large dia- log corpus in the right language and related domain. We leverage Simulated Multiple Reference Training (SMRT; Khayrallah et al., 2020) to over- come sparse dialog data. SMRT uses a word- level knowledge distillation-inspired objective and a paraphraser to simulate multiple references per training example. Khayrallah et al. introduce SMRT for machine translation (MT) and simulate training on all translations for a source sentence, assuming: (1) all paraphrases of a target are trans- lations of the source; and (2) all translations of the source are paraphrases of the target. (1) is true for dialog, but (2) is not—valid chatbot responses vary in meaning. SMRT captures syntactic diversity though it cannot represent all semantic variations. prompt: Study, study, study. I want to learn a lot. response: You are going to take courses? paraphrases: you You are going to take courses? going classes? to take classes? you take courses? Will Do classes? courses? you plan to take planning to take Are courses? taking on Table 1: A DailyDialog training pair and paraphrases. The tree of paraphrases includes some ........ possible ............ paraphrases of the original promt, a sampled path and some of the other tokens also considered in the training objective. We apply SMRT to chatbots and find that it: (1) improves human and automatic quality scores; (2) improves lexical diversity; (3) performs as well as pretraining in human evaluation with better per- formance on automatic measures of diversity and quality. 2 Method We model the non-task-oriented dialog system (chatbot) task as conditional language modeling. These models are typically trained using Nega- tive Log Likelihood (NLL) with respect to a single reference. An alternative approach is Knowledge Distillation (Hinton et al., 2015; Kim and Rush, 2016) which assumes access to a teacher distribu- tion (q(y | x)) and minimizes the cross entropy …

Content truncated to 500 words