Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2481–2491 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2481 Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation Chao Zhao†, Marilyn Walker‡ and Snigdha Chaturvedi† † Department of Computer Science, University of North Carolina at Chapel Hill ‡ Natural Language and Dialog Systems Lab, University of California, Santa Cruz {zhaochao, snigdha}@cs.unc.edu mawalker@ucsc.edu Abstract Generating sequential natural language de- scriptions from graph-structured data (e.g., knowledge graph) is challenging, partly be- cause of the structural differences between the input graph and the output text. Hence, pop- ular sequence-to-sequence models, which re- quire serialized input, are not a natural fit for this task. Graph neural networks, on the other hand, can better encode the input graph but broaden the structural gap between the en- coder and decoder, making faithful generation difficult. To narrow this gap, we propose DUA- LENC, a dual encoding model that can not only incorporate the graph structure, but can also cater to the linear structure of the out- put text. Empirical comparisons with strong single-encoder baselines demonstrate that dual encoding can significantly improve the quality of the generated text. 1 Introduction Data-to-text generation aims to create natural lan- guage text to describe the input data (Reiter and Dale, 2000). Here we focus on structured text in- put in a particular form such as a tree or a graph. Figure 1 shows an example where the input data is a mini knowledge graph, and the output text is its corresponding natural language description. Gener- ating text from such data is helpful for many NLP tasks, such as question answering and dialogue (He et al., 2017; Liu et al., 2018; Moon et al., 2019). During generation, the structure of the data as well as the content inside the structure jointly de- termine the generated text. For example, the direc- tion of the edge “capital” in Figure 1 determines that “London is the capital of U.K.” is an accurate description, but not vice versa. Current genera- tion methods are based on sequence-to-sequence (Seq2Seq) encoder-decoder architecture (Sutskever et al., 2014), which requires the input data to be Figure 1: Illustration of the WebNLG challenge: the source data is an RDF graph and the target output is a text description of the graph. serialized as a sequence, resulting in a loss of struc- tural information. Recent research has shown the utility of incorpo- rating structural information during generation. By replacing the sequential encoder with a structure- aware graph encoder, such as a graph convolu- tional network (GCNs) (Kipf and Welling, 2017) or graph-state LSTMs (Song et al., 2018), the re- sulting graph-to-sequence (Graph2Seq) methods can encode the structural information of the input and thus outperform Seq2Seq models on certain tasks. However, these architectures broaden the structural gap between the encoder and decoder. That is, while the encoder receives the input data as a graph, the decoder has to create the output text as a linear chain …

Content truncated to 500 words