EARL: Informative Knowledge-Grounded Conversation Generation with Entity-Agnostic Representation Learning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2383–2395 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2383 EARL: Informative Knowledge-Grounded Conversation Generation with Entity-Agnostic Representation Learning Hao Zhou1,2, Minlie Huang1∗, Yong Liu3, Wei Chen3, Xiaoyan Zhu1 1The CoAI group, DCST, State Key Lab of Intelligent Technology and Systems, 1Beijing National Research Center for Information Science and Technology, 1Tsinghua University, Beijing 100084, China 2Pattern Recognition Center, WeChat AI, Tencent Inc., China 3Sogou Inc., Beijing 100084, China tuxchow@gmail.com, aihuang@tsinghua.edu.cn Abstract Generating informative and appropriate re- sponses is challenging but important for building human-like dialogue systems. Al- though various knowledge-grounded conversa- tion models have been proposed, these mod- els have limitations in utilizing knowledge that infrequently occurs in the training data, not to mention integrating unseen knowledge into conversation generation. In this paper, we pro- pose an Entity-Agnostic Representation Learn- ing (EARL) method to introduce knowledge graphs to informative conversation generation. Unlike traditional approaches that parameter- ize the specific representation for each en- tity, EARL utilizes the context of conversa- tions and the relational structure of knowledge graphs to learn the category representation for entities, which is generalized to incorporating unseen entities in knowledge graphs into con- versation generation. Automatic and manual evaluations demonstrate that our model can generate more informative, coherent, and natu- ral responses than baseline models. 1 Introduction Generating informative and appropriate responses is vital for the success of human-like dialogue sys- tems. To this end, there has been a rising ten- dency in enhancing conversation models with ex- ternal knowledge recently, which is well-known as the knowledge-grounded conversation model (Ghazvininejad et al., 2018; Zhou et al., 2018; Di- nan et al., 2019). Several studies incorporate un- structured texts, such as web pages (Ghazvininejad et al., 2018) and Wikipedia articles (Dinan et al., 2019), as the external knowledge to generate infor- mative responses. Some work introduces structured knowledge, e.g. the knowledge graph (Zhou et al., 2018) to generate knowledge enhanced conversa- tions. ∗ Corresponding author: Minlie Huang. KG # Entities # Triples # Relations Freebase 40M 637M 35,000 Wikidata 18M 66M 1,632 ConceptNet 8M 21M 36 Table 1: Statistics of some widely used knowledge graphs (KG, Knowledge Graph; M, million). Prior studies adopt either pre-trained knowledge graph embeddings (Zhou et al., 2018), e.g. TransE (Bordes et al., 2013), word embeddings (Wu et al., 2019), or adjacency matrix (Tuan et al., 2019) to model entities and relations in knowledge graphs and incorporate them to conversation generation. These models face two major challenges when ap- plied to introduce large-scale knowledge graphs. First, there is a significant gap in representations between knowledge and text (Buitelaar and Cimi- ano, 2008; Zhou et al., 2018), which requires model training to apply knowledge in conversation gen- eration based on different knowledge representa- tions. However, the training corpus of knowledge- grounded conversations only contains a small sub- set of entities for applying knowledge, while the large-scale untrained entities are difficult to be uti- lized due to the gap between their representations. …

Content truncated to 500 words