AttnIO: Knowledge Graph Exploration with In-and-Out Attention Flow for Knowledge-Grounded Dialogue

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3484–3497, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3484 AttnIO: Knowledge Graph Exploration with In-and-Out Attention Flow for Knowledge-Grounded Dialogue Jaehun Jung1,2 Bokyung Son1,3* Sungwon Lyu1* 1Kakao Enterprise Corporation 2Department of Computer Science, Seoul National University 3Department of Linguistics, Seoul National University sharkmir1@snu.ac.kr, {meta.mon, james.ryu}@kakaoenterprise.com Abstract Retrieving the proper knowledge relevant to conversational context is an important chal- lenge in dialogue systems, to engage users with more informative response. Several re- cent works propose to formulate this knowl- edge selection problem as a path traversal over an external knowledge graph (KG), but show only a limited utilization of KG struc- ture, leaving rooms of improvement in perfor- mance. To this effect, we present AttnIO, a new dialog-conditioned path traversal model that makes a full use of rich structural infor- mation in KG based on two directions of at- tention flows. Through the attention flows, At- tnIO is not only capable of exploring a broad range of multi-hop knowledge paths, but also learns to flexibly adjust the varying range of plausible nodes and edges to attend depending on the dialog context. Empirical evaluations present a marked performance improvement of AttnIO compared to all baselines in OpenDi- alKG dataset. Also, we find that our model can be trained to generate an adequate knowledge path even when the paths are not available and only the destination nodes are given as label, making it more applicable to real-world dia- logue systems. 1 Introduction One of the milestone challenges in conversational AI is to engage users with a more informative and knowledgeable response, rather than merely out- putting generic sentences. For instance, given a user’s utterance saying “I’m a big fan of Steven Spielberg”, it would be more engaging to respond “My favorite movie is his science fiction film A.I.”, rather than “I like him too.”. An external source of knowledge such as knowledge graph (KG) can *Equal contribution. play a crucial role here, as it could help the con- versational agent with informative paths, such as “Steven Spielberg, directed, A.I., has genre, Sci- ence Fiction”. The above mentioned motivation gave rise to a conspicuous need for path retrieval model on KG, which can learn to traverse a path consisting of proper entities and relations to mention in the next response, given the dialog context. Previous approaches to this knowledge selection problem rely on either an RL-based agent (Liu et al., 2019) or a recurrent decoder (Moon et al., 2019), which greedily selects the most proper entity to traverse regarding its previous decision. Despite their nov- elty, we find several rooms of improvement from the previous works, to move toward a more fine- grained modeling of knowledge path retrieval for dialogue systems. First, a model could make use of the rich rela- tional information residing at the neighborhood of each node on KG. Typically, the number of entities in KG is large, while the numbers of each entity’s usage in actual …

Content truncated to 500 words