Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2031–2043 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2031 Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs Houyu Zhang1 ∗† Zhenghao Liu2∗ Chenyan Xiong3 Zhiyuan Liu2 1Department of Computer Science, Brown University, Providence, USA 2Department of Computer Science and Technology, Tsinghua University, Beijing, China Institute for Artificial Intelligence, Tsinghua University, Beijing, China State Key Lab on Intelligent Technology and Systems, Tsinghua University, Beijing, China 3Microsoft Research AI, Redmond, USA Abstract Human conversations naturally evolve around related concepts and scatter to multi-hop con- cepts. This paper presents a new conversation generation model, ConceptFlow, which lever- ages commonsense knowledge graphs to ex- plicitly model conversation flows. By ground- ing conversations to the concept space, Con- ceptFlow represents the potential conversa- tion flow as traverses in the concept space along commonsense relations. The traverse is guided by graph attentions in the con- cept graph, moving towards more meaning- ful directions in the concept space, in or- der to generate more semantic and informa- tive responses. Experiments on Reddit con- versations demonstrate ConceptFlow’s effec- tiveness over previous knowledge-aware con- versation models and GPT-2 based models while using 70% fewer parameters, confirm- ing the advantage of explicit modeling con- versation structures. All source codes of this work are available at https://github.com/ thunlp/ConceptFlow. 1 Introduction The rapid advancements of language modeling and natural language generation (NLG) techniques have enabled fully data-driven conversation models, which directly generate natural language responses for conversations (Shang et al., 2015; Vinyals and Le, 2015; Li et al., 2016b). However, it is a com- mon problem that the generation models may de- generate dull and repetitive contents (Holtzman et al., 2019; Welleck et al., 2019), which, in con- versation assistants, leads to off-topic and useless responses. (Tang et al., 2019; Zhang et al., 2018; Gao et al., 2019). Conversations often develop around Knowledge. A promising way to address the degeneration prob- ∗Indicates equal contribution. †Part of work is conducted at Tsinghua University. Original Graph chat based future steam paper class talk text hope plan voice book write faith dream bag card word idea water dream hope faith future talk text write card word chat voice POST： chat based on knowledge is the future Response：yeah it ’s not a dream to have a talk with robot Zero-hop Concept One-hop Concept Two-hop Concept Figure 1: An Example of Concept Shift in a Conver- sation. Darker green indicates higher relevance and wider arrow indicates stronger concept shift (captured by ConceptFlow). lem is to ground conversations with external knowl- edge (Xing et al., 2017), such as open-domain knowledge graph (Ghazvininejad et al., 2018), com- monsense knowledge base (Zhou et al., 2018a), or background documents (Zhou et al., 2018b). Re- cent research leverages such external knowledge by using them to ground conversations, integrat- ing them as additional representations, and then generating responses conditioned on both the texts and the grounded semantics (Ghazvininejad et al., 2018; Zhou et al., 2018a,b). …

Content truncated to 500 words