Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4604–4616 July 9-14, 2023 ©2023 Association for Computational Linguistics Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation Chen Tang1, Hongbo Zhang2, Tyler Loakman2, Chenghua Lin2∗and Frank Guerin1 Department of Computer Science, The University of Sheffield, UK Department of Computer Science, The University of Surrey, UK {chen.tang,f.guerin}@surrey.ac.uk {hzhang183,tcloakman1,c.lin}@sheffield.ac.uk Abstract Incorporating external graph knowledge into neural chatbot models has been proven effec- tive for enhancing dialogue generation. How- ever, in conventional graph neural networks (GNNs), message passing on a graph is inde- pendent from text, resulting in the graph rep- resentation hidden space differing from that of the text. This training regime of existing mod- els therefore leads to a semantic gap between graph knowledge and text. In this study, we pro- pose a novel framework for knowledge graph enhanced dialogue generation. We dynamically construct a multi-hop knowledge graph with pseudo nodes to involve the language model in feature aggregation within the graph at all steps. To avoid the semantic biases caused by learning on vanilla subgraphs, the proposed framework applies hierarchical graph attention to aggregate graph features on pseudo nodes and then attains a global feature. Therefore, the framework can better utilise the heteroge- neous features from both the post and exter- nal graph knowledge. Extensive experiments demonstrate that our framework outperforms state-of-the-art (SOTA) baselines on dialogue generation. Further analysis also shows that our representation learning framework can fill the semantic gap by coagulating representa- tions of both text and graph knowledge. More- over, the language model also learns how to better select knowledge triples for a more in- formative response via exploiting subgraph patterns within our feature aggregation pro- cess. Our code and resources are available at https://github.com/tangg555/SaBART. 1 Introduction Recent years have seen a surge of interest in devel- oping chatbots with the facilitation of large-scale knowledge (Ni et al., 2022). As a highly expressive data format, Knowledge Graphs (e.g. ConceptNet and DBpedia), which include world facts, are con- sidered to be a key factor in building an effective *Corresponding author. 𝒉𝟏 ′ 𝒈 𝟏 𝒈 𝟐 Post 𝑳𝑴 (a) GNN + Static Graph 𝒉𝟐 ′ 𝑳𝑴𝑬𝒏𝒄. (b) Ours. + Dynamic Graph √ ∫ √ ∫ 𝒈 𝟏 ′ 𝒈 𝟐 ′ 𝑮′ Embed Graph Aggregation Post √ ∫ 𝑳𝑴𝑬𝒏𝒄. 𝑯′ √ ∫ 𝒉𝟐 ′ Generation Selection 𝑳𝑴𝑫𝒆𝒄. G Figure 1: Conventional GNNs vs. Ours. dialogue generation system (Zhou et al., 2018). In order to incorporate graph-structured knowledge, a range of Graph Neural Networks such as Graph At- tention Networks (GATs) (Velickovic et al., 2017; Brody et al., 2021) and Graph Convolutional Net- works (GCNs) (Kipf and Welling, 2016) have been proposed to learn representations of the topologi- cal structure of the knowledge graph via message passing between entities. In open-domain dialogue generation, these GNNs are further embedded into generative frameworks to feed graph knowledge features into the language models (LMs). Despite prior success in leveraging graph knowl- edge with graph neural …

Content truncated to 500 words