Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs

Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs Leonardo F. R. Ribeiro†, Yue Zhang‡, Claire Gardent§ and Iryna Gurevych† †Research Training Group AIPHES and UKP Lab, Technische Universit¨at Darmstadt ‡School of Engineering, Westlake University, §CNRS/LORIA, Nancy, France ribeiro@aiphes.tu-darmstadt.de, yue.zhang@wias.org.cn claire.gardent@loria.fr, gurevych@ukp.informatik.tu-darmstadt.de Abstract Recent graph-to-text models generate text from graph-based data using either global or local aggregation to learn node representa- tions. Global node encoding allows explicit communication between two distant nodes, thereby neglecting graph topology as all nodes are directly connected. In contrast, local node encodingconsiders the relations between neigh- bor nodes capturing the graph structure, but it can fail to capture long-range relations. In this work, we gather both encoding strategies, pro- posing novel neural models that encode an input graph combining both global and local node contexts, in order to learn better contextu- alized node embeddings. In our experiments, we demonstrate that our approaches lead to significant improvements on two graph-to- text datasets achieving BLEU scores of 18.01 on the AGENDA dataset, and 63.69 on the WebNLG dataset for seen categories, outper- forming state-of-the-art models by 3.7 and 3.1 points, respectively.1 1 Introduction Graph-to-text generation refers to the task of gen- erating natural language text from input graph structures, which can be semantic representations (Konstas et al., 2017) or knowledge graphs (KGs) (Gardent et al., 2017; Koncel-Kedziorski et al., 2019). Whereas most recent work (Song et al., 2018; Ribeiro et al., 2019; Guo et al., 2019) fo- cuses on generating sentences, a more challenging and interesting scenario emerges when the goal is to generate multisentence texts. In this context, in addition to sentence generation, document plan- ning needs to be handled: The input needs to be mapped into several sentences; sentences need to 1Codeis available at https://github.com/UKPLab/ kg2text. be ordered and connected using appropriate dis- course markers; and inter-sentential anaphora and ellipsis may need to be generated to avoid repeti- tion. In this paper, we focus on generating texts rather than sentences where the output are short texts (Gardent et al., 2017) or paragraphs (Koncel- Kedziorski et al., 2019). A key issue in neural graph-to-text generation is how to encode the input graphs. The basic idea is to incrementally compute node representations by aggregating structural context information. To this end, two main approaches have been proposed: (i) models based on local node aggregation, usually built upon Graph Neural Networks (GNNs) (Kipf and Welling, 2017; Hamilton et al., 2017) and (ii) models that leverage global node aggregation. Systems that adopt global encoding strategies are typically based on Transformers (Vaswani et al. 2017), using self-attention to compute a node representation based on allnodes in the graph. This approach enjoys the advantage of a large node con- text range, but neglects the graph topology by effectively treating every node as being connected to all the others in the graph. In contrast, models based on local aggregation learn the representation of each node based on its adjacent nodes as defined in the input graph. This approach effect- ively …

Content truncated to 500 words