Modeling Semantic Relationship in Multi-turn Conversations with Hierarchical Latent Variables

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5497–5502 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5497 Modeling Semantic Relationship in Multi-turn Conversations with Hierarchical Latent Variables Lei Shen1,2 Yang Feng1,2∗ Haolan Zhan2,3 1Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China 2University of Chinese Academy of Sciences, Beijing, China 3State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, China {shenlei17z, fengyang}@ict.ac.cn zhanhl@ios.ac.cn Abstract Multi-turn conversations consist of complex semantic structures, and it is still a challenge to generate coherent and diverse responses given previous utterances. It’s practical that a conversation takes place under a background, meanwhile, the query and response are usu- ally most related and they are consistent in topic but also different in content. However, little work focuses on such hierarchical rela- tionship among utterances. To address this problem, we propose a Conversational Seman- tic Relationship RNN (CSRR) model to con- struct the dependency explicitly. The model contains latent variables in three hierarchies. The discourse-level one captures the global background, the pair-level one stands for the common topic information between query and response, and the utterance-level ones try to represent differences in content. Experimen- tal results show that our model significantly improves the quality of responses in terms of fluency, coherence and diversity compared to baseline methods. 1 Introduction Inspired by the observation that real-world human conversations are usually multi-turn, some stud- ies have focused on multi-turn conversations and taken context (history utterances in previous turns) into account for response generation. How to model the relationship between the response and context is essential to generate coherent and logi- cal conversations. Currently, the researchers em- ploy some hierarchical architectures to model the relationship. Serban et al. (2016) use a context RNN to integrate historical information, Tian et al. (2017) sum up all utterances weighted by the sim- ilarity score between an utterance and the query, while Zhang et al. (2018) apply attention mecha- nism on history utterances. Besides, Xing et al. ∗Corresponding Author (2018) add a word-level attention to capture fine- grained features. In practice, we usually need to understand the meaning of utterances and capture their semantic dependency, not just word-level alignments (Luo et al., 2018). As shown in Table 1, this short con- versation is about speaker A asks the current sit- uation of speaker B. At the beginning, they talk about B’s position. Then in the last two utter- ances, both speakers think about the way for B to come back. A mentions “umbrella”, while B wants A to “pick him/her up”. What’s more, there is no “word-to-word” matching in query and re- sponse. Unfortunately, the aforementioned hierar- chical architectures do not model the meaning of each utterance explicitly and has to summarize the meaning of utterances on the fly during generat- ing the response, and hence there is no guarantee that the inferred meaning is adequate to the origi- nal utterance. …

Content truncated to 500 words