DyKgChat: Benchmarking Dialogue Generation Grounding on Dynamic Knowledge Graphs

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1855–1865, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1855 DyKgChat: Benchmarking Dialogue Generation Grounding on Dynamic Knowledge Graphs Yi-Lin Tuan Yun-Nung Chen Hung-yi Lee National Taiwan University, Taipei, Taiwan pascaltuan@gmail.com y.v.chen@ieee.org hungyilee@ntu.edu.tw Abstract Data-driven, knowledge-grounded neural con- versation models are capable of generating more informative responses. However, these models have not yet demonstrated that they can zero-shot adapt to updated, unseen knowl- edge graphs. This paper proposes a new task about how to apply dynamic knowl- edge graphs in neural conversation model and presents a novel TV series conversation corpus (DyKgChat) for the task. Our new task and corpus aids in understanding the influence of dynamic knowledge graphs on responses gen- eration. Also, we propose a preliminary model that selects an output from two networks at each time step: a sequence-to-sequence model (Seq2Seq) and a multi-hop reasoning model, in order to support dynamic knowl- edge graphs. To benchmark this new task and evaluate the capability of adaptation, we intro- duce several evaluation metrics and the exper- iments show that our proposed approach out- performs previous knowledge-grounded con- versation models. The proposed corpus and model can motivate the future research direc- tions1. 1 Introduction In the chit-chat dialogue generation, neural con- versation models (Sutskever et al., 2014; Sordoni et al., 2015; Vinyals and Le, 2015) have emerged for its capability to be fully data-driven and end- to-end trained. While the generated responses are often reasonable but general (without useful information), recent work proposed knowledge- grounded models (Eric et al., 2017; Ghazvinine- jad et al., 2018; Zhou et al., 2018b; Qian et al., 2018) to incorporate external facts in an end-to- end fashion without hand-crafted slot filling. Ef- fectively combining text and external knowledge 1The data and code are available in https://github. com/Pascalson/DyKGChat. ServantOf EnemyOf TitleOf ResidenceOf ResidenceOf LoverOf Zhen, Huan: Thanks for Consort Jing's concern. TitleOf EnemyOf Zhen, Huan: Thanks for Consort Hua's concern. Enemy of Jin-Xi is Feng, Ruozhao Enemy of Jin-Xi is Nian, Shilan Knowledge Graph Jin-Xi 槿汐 Zhen, Huan 甄嬛 Nian, Shilan 年世蘭 Feng, Ruozhao 馮若昭 Consort Jing 敬妃 Consort Hua 華妃 Jin-Xi: A consort just passed by and asked about you. Training Testing Input Out Out (an unseen relation) Figure 1: An example of an ideal conversation model with dynamic knowledge graphs. graphs have also been a crucial topic in ques- tion answering (Yin et al., 2016; Hao et al., 2017; Levy et al., 2017; Sun et al., 2018; Das et al., 2019). Nonetheless, prior work rarely analyzed the model capability of zero-shot adaptation to dy- namic knowledge graphs, where the states/entities and their relations are temporal and evolve as a single time scale process. For example, as shown in Figure 1, the entity Jin-Xi was originally related to the entity Feng, Ruozhao with the type Ene- myOf, but then evolved to be related to the entity Nian, Shilan. The …

Content truncated to 500 words