A Diversity-Promoting Objective Function for Neural Conversation Models

Proceedings of NAACL-HLT 2016, pages 110–119, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics A Diversity-Promoting Objective Function for Neural Conversation Models Jiwei Li1∗ Michel Galley2 Chris Brockett2 Jianfeng Gao2 Bill Dolan2 1Stanford University, Stanford, CA, USA jiweil@stanford.edu 2Microsoft Research, Redmond, WA, USA {mgalley,chrisbkt,jfgao,billdol}@microsoft.com Abstract Sequence-to-sequence neural network models for generation of conversational responses tend to generate safe, commonplace responses (e.g., I don’t know) regardless of the input. We sug- gest that the traditional objective function, i.e., the likelihood of output (response) given input (message) is unsuited to response generation tasks. Instead we propose using Maximum Mu- tual Information (MMI) as the objective func- tion in neural models. Experimental results demonstrate that the proposed MMI models produce more diverse, interesting, and appro- priate responses, yielding substantive gains in BLEU scores on two conversational datasets and in human evaluations. 1 Introduction Conversational agents are of growing importance in facilitating smooth interaction between humans and their electronic devices, yet conventional dialog sys- tems continue to face major challenges in the form of robustness, scalability and domain adaptation. At- tention has thus turned to learning conversational pat- terns from data: researchers have begun to explore data-driven generation of conversational responses within the framework of statistical machine transla- tion (SMT), either phrase-based (Ritter et al., 2011), or using neural networks to rerank, or directly in the form of sequence-to-sequence (SEQ2SEQ) models (Sordoni et al., 2015; Vinyals and Le, 2015; Shang et al., 2015; Serban et al., 2015; Wen et al., 2015). SEQ2SEQ models offer the promise of scalability and language-independence, together with the capacity * The entirety of this work was conducted at Microsoft. to implicitly learn semantic and syntactic relations between pairs, and to capture contextual dependen- cies (Sordoni et al., 2015) in a way not possible with conventional SMT approaches (Ritter et al., 2011). An engaging response generation system should be able to output grammatical, coherent responses that are diverse and interesting. In practice, how- ever, neural conversation models tend to generate trivial or non-committal responses, often involving high-frequency phrases along the lines of I don’t know or I’m OK (Sordoni et al., 2015; Serban et al., 2015; Vinyals and Le, 2015). Table 1 illustrates this phenomenon, showing top outputs from SEQ2SEQ models. All the top-ranked responses are generic. Responses that seem more meaningful or specific can also be found in the N-best lists, but rank much lower. In part at least, this behavior can be ascribed to the relative frequency of generic responses like I don’t know in conversational datasets, in contrast with the relative sparsity of more contentful alterna- tive responses.1 It appears that by optimizing for the likelihood of outputs given inputs, neural mod- els assign high probability to “safe” responses. This objective function, common in related tasks such as machine translation, may be unsuited to generation tasks involving intrinsically diverse outputs. Intu- itively, it seems desirable to take into account not only the dependency of responses on messages, but also the inverse, the likelihood that …

Content truncated to 500 words