Augmenting Knowledge-grounded Conversations with Sequential Knowledge Transition

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5621–5630 June 6–11, 2021. ©2021 Association for Computational Linguistics 5621 Augmenting Knowledge-grounded Conversations with Sequential Knowledge Transition Haolan Zhan1∗, Hainan Zhang2†, Hongshen Chen2, Zhuoye Ding2 Yongjun Bao2 and Yanyan Lan3 1 Institute of Software, Chinese Academy of Sciences, Beijing, China 2Data Science Lab, JD.com, Beijing, China 3Institute for AI Industry Research, Tsinghua University, Beijing, China zhanhaolan316@gmail.com, zhanghainan6@jd.com, ac@chenhongshen.com, dingzhuoye@jd.com, lanyanyan@tsinghua.edu.cn Abstract Knowledge data are massive and widespread in the real-world, which can serve as good ex- ternal sources to enrich conversations. How- ever, in knowledge-grounded conversations, current models still lack the fine-grained con- trol over knowledge selection and integra- tion with dialogues, which finally leads to the knowledge-irrelevant response generation problems: 1) knowledge selection merely re- lies on the dialogue context, ignoring the inher- ent knowledge transitions along with conversa- tion flows; 2) the models often over-fit during training, resulting with incoherent response by referring to unrelated tokens from specific knowledge content in the testing phase; 3) al- though response is generated upon the dia- logue history and knowledge, the models of- ten tend to overlook the selected knowledge, and hence generates knowledge-irrelevant re- sponse. To address these problems, we pro- posed to explicitly model the knowledge tran- sition in sequential multi-turn conversations by abstracting knowledge into topic tags. Be- sides, to fully utilizing the selected knowl- edge in generative process, we propose pre- training a knowledge-aware response gener- ator to pay more attention on the selected knowledge. In particular, a sequential knowl- edge transition model equipped with a pre- trained knowledge-aware response generator (SKT-KG) formulates the high-level knowl- edge transition and fully utilizes the limited knowledge data. Experimental results on both structured and unstructured knowledge- grounded dialogue benchmarks indicate that our model achieves better performance over baseline models. 1 Introduction Knowledge-grounded conversations (Long et al., 2017; Liu et al., 2018; Niu et al., 2019; Xu et al., 2020), aiming at improving the informativeness ∗Work done at Data Science Lab, JD.com. †Corresponding author. Do you like Aries actors ? Of course, I am the Aries! Chao Wu's astrological sign is also the Aries. I know him, his blood type is O. Is it good? What the main theme? it tells a story about twin flowers. Conversations Knowledge Transition {actor, astrological sign, Aries} {actor, astrological sign, Aries} {Chao Wu, astrological sign, Aries} {Lunar Eclipse, review, twin flowers} {Chao Wu, blood typology, O type} Really? I just know one of his movies Lunar Eclipse. Great, I will see it. {Lunar Eclipse, review, twin flowers} {Chao Wu, masterpiece, Lunar Eclipse} {Chao Wu, masterpiece, Lunar Eclipse} Figure 1: The example from the DuConv dataset (Wu et al., 2019), shows the knowledge transition in real di- alogue. and specificity of dialogue generation by exploit- ing external knowledge sources, has attracted much attention as a potential solution to relieve the com- mon response problem (Li et al., 2015; Zhang et al., 2018a; Ren et al., 2020) …

Content truncated to 500 words