Towards Implicit Content-Introducing for Generative Short-Text Conversation Systems

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2190–2199 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Towards Implicit Content-Introducing for Generative Short-Text Conversation Systems Lili Yao1, Yaoyuan Zhang1, Yansong Feng1, Dongyan Zhao1,2 and Rui Yan1,2 ∗ 1Institute of Computer Science and Technology, Peking University, Beijing, China 2Beijing Institute of Big Data Research, Beijing, China {yaolili,zhang yaoyuan,fengyansong,zhaody,ruiyan}@pku.edu.cn Abstract The study on human-computer conversa- tion systems is a hot research topic nowa- days. One of the prevailing method- s to build the system is using the gen- erative Sequence-to-Sequence (Seq2Seq) model through neural networks. Howev- er, the standard Seq2Seq model is prone to generate trivial responses. In this pa- per, we aim to generate a more meaning- ful and informative reply when answering a given question. We propose an implicit content-introducing method which incor- porates additional information into the Se- q2Seq model in a flexible way. Specifical- ly, we fuse the general decoding and the auxiliary cue word information through our proposed hierarchical gated fusion u- nit. Experiments on real-life data demon- strate that our model consistently outper- forms a set of competitive baselines in terms of BLEU scores and human evalu- ation. 1 Introduction To establish a conversation system with adequate artificial intelligence is a long-cherished goal for researchers and practitioners. In particular, auto- matic conversation systems in open domains are attracting increasing attention due to its wide ap- plications, such as virtual assistants and chatbot- s. In open domains, researchers mainly focus on data-driven approaches, since the diversity and un- certainty make it impossible to prepare the inter- action logic and domain knowledge. Basically, there are two mainstream ways to build an open- domain conversation system: 1) to search pre- established database for candidate responses by ∗Corresponding author: ruiyan@pku.edu.cn query retrieval (Isbell et al., 2000; Wang et al., 2013; Yan et al., 2016; Song et al., 2016), and 2) to generate a new, tailored utterance given the user- issued query (Shang et al., 2015; Vinyals and Le, 2015; Serban et al., 2016; Mou et al., 2016; Song et al., 2016). In these studies, generation-based conversation systems have shown impressive po- tential. Especially, the Sequence-to-Sequence (Se- q2Seq) model (Sutskever et al., 2014) based on neural networks has been extensively used in prac- tice; the idea is to encode a query as a vector and to decode the vector into a reply. Inspired by (Mou et al., 2016), we mainly focus on the generative short-text conversation without context informa- tion. Despite this, the performance of Seq2Seq generation-based conversation systems is far from satisfactory because its generation process is not controllable; it responses to a query according to the pattern learned from the training corpus. As a result, the system is likely to generate an un- expected reply even with little semantics, e.g, “I don’t know” and “Okay” due to the high frequency of these patterns in training data (Li et al., 2016a; Mou et al., 2016). To address this issue, Li et al. (2016a) …

Content truncated to 500 words