Modeling Multi-turn Conversation with Deep Utterance Aggregation

Proceedings of the 27th International Conference on Computational Linguistics, pages 3740–3752 Santa Fe, New Mexico, USA, August 20-26, 2018. 3740 Modeling Multi-turn Conversation with Deep Utterance Aggregation Zhuosheng Zhang1,2,∗, Jiangtong Li1,2,3,∗, Pengfei Zhu1,2,5, Hai Zhao1,2,†, Gongshen Liu4 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China 3College of Zhiyuan, Shanghai Jiao Tong University, China 4School of Cyber Security, Shanghai Jiao Tong University, China 5School of Computer Science and Software Engineering, East China Normal University, China {zhangzs, keep moving-lee}@sjtu.edu.cn, 10152510190@stu.ecnu.edu.cn, zhaohai@cs.sjtu.edu.cn, lgsheng@sjtu.edu.cn Abstract Multi-turn conversation understanding is a major challenge for building intelligent dialogue sys- tems. This work focuses on retrieval-based response matching for multi-turn conversation whose related work simply concatenates the conversation utterances, ignoring the interactions among previous utterances for context modeling. In this paper, we formulate previous utterances into context using a proposed deep utterance aggregation model to form a fine-grained context rep- resentation. In detail, a self-matching attention is first introduced to route the vital information in each utterance. Then the model matches a response with each refined utterance and the fi- nal matching score is obtained after attentive turns aggregation. Experimental results show our model outperforms the state-of-the-art methods on three multi-turn conversation benchmarks, including a newly introduced e-commerce dialogue corpus. 1 Introduction Human-computer interactive systems are booming due to their promising potentials and alluring com- mercial values (Qiu et al., 2017; Cui et al., 2017; Yan et al., 2017; Huang et al., 2018; Jia and Zhao, 2014). With the development of neural models (Zhang et al., 2018c; He et al., 2018; Li et al., 2018; Cai et al., 2018; Zhang and Zhao, 2018), building an intelligent dialogue system as our personal assistant or chat companion, is no longer a fantasy, among which multi-turn natural language understanding still keeps extremely challenging, requiring the system to comprehend the conversation context and reply in an informative and coincident manner. Multi-turn conversation modeling plays a key role in dialogue systems, either for generation-based (Serban et al., 2017b; Serban et al., 2017a; Zhou et al., 2017; Wu et al., 2018) or retrieval-based ones (Wu et al., 2017; Zhou et al., 2016) in which the latter is the focus of this paper. A natural approach for multi-turn modeling is simply concatenating the context utterances (Lowe et al., 2015; Yan et al., 2016). However, this will introduce much noise since previous utterances as the context is lengthy and redundant. The gist is to identify pertinent information in previous utterances and properly model the utterance relationships to ensure conversation consistency. To avoid unnecessary information loss, (Wu et al., 2017) matches a response with each utterance in the context, paying little attention on distrinct importance of each utterance and also failing to touch internal semantics inside utterances. In fact, the relevance of each utterance to the supposed response usually varies. As shown in Figure 1, the last utterance in a conversation empirically conveys the user …

Content truncated to 500 words