Dialog Generation Using Multi-Turn Reasoning Neural Networks

Proceedings of NAACL-HLT 2018, pages 2049–2059 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Dialog Generation Using Multi-turn Reasoning Neural Networks Xianchao Wu1, Ander Mart´ınez2∗, Momo Klyen1 1A.I.&Research, Microsoft Development Co., Ltd. 2 Graduate School of Information Science, Nara Institute of Science and Technology {xiancwu, momokl}@microsoft.com, ander.martinez.zy4@is.naist.jp Abstract In this paper, we propose a generalizable di- alog generation approach that adapts multi- turn reasoning, one recent advancement in the field of document comprehension, to gener- ate responses (“answers”) by taking current conversation session context as a “document” and current query as a “question”. The major idea is to represent a conversation session into memories upon which attention-based mem- ory reading mechanism can be performed mul- tiple times, so that (1) user’s query is properly extended by contextual clues and (2) optimal responses are step-by-step generated. Consid- ering that the speakers of one conversation are not limited to be one, we separate the single memory used for document comprehension into different groups for speaker-specific topic and opinion embedding. Namely, we utilize the queries’ memory, the responses’ memory, and their unified memory, following the time sequence of the conversation session. Experi- ments on Japanese 10-sentence (5-round) con- versation modeling show impressive results on how multi-turn reasoning can produce more diverse and acceptable responses than state- of-the-art single-turn and non-reasoning base- lines. 1 Introduction Dialogue systems such as chatbots are a thriving topic that is attracting increasing attentions from researchers (Sordoni et al., 2015; Serban et al., 2016; Li et al., 2015; Wen et al., 2016). Recent achievements, such as deep neural networks for text generating, user profiling (Li et al., 2014), and natural language understanding, have accelerated the progresses of this field, which was historically approached by conventional rule-based and/or sta- tistical response ranking strategies. ∗Work done when Ander was an intern in Microsoft. Wu and Ander contributed equally to this paper. Response ranking models retrieve the most suit- able response(s) from a fixed set of (question, answer) pairs given a dialogue context and cur- rent query from a user (Banchs and Li, 2012; Lowe et al., 2015). Learning-to-rank approaches were applied to compute the similarity scores of between (query, context) and indexed candidate (question, answer) pairs to return the optimal “an- swer” to the user. These ranking-based retrieval strategies have been well-applied as an impor- tant approach to dialogue systems, yet the set of scripted responses are limited and are short at gen- eralization. On the other hand, statistical machine translation (SMT) systems have been applied to dialogue systems (Ritter et al., 2011), taking user’s query as a source language sentence and the chat- bot’s response as a target language sentence. La- beled data for learning-to-ranking training will not be necessary anymore and all we need is the large- scale (question, answer) pairs. The sequence-to-sequence model proposed in (Sutskever et al., 2014) applied end-to-end train- ing of neural networks to text generation. This model, further enhanced by an attention mecha- nism (Bahdanau et al., 2014), was …

Content truncated to 500 words