Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning

Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning Zhijiang Guo1∗, Yan Zhang1∗, Zhiyang Teng1,2, Wei Lu1 1Singapore University of Technology and Design 8 Somapah Road, Singapore, 487372 2School of Engineering, Westlake University, China {zhijiang guo,yan zhang,zhiyang teng}@mymail.sutd.edu.sg tengzhiyang@westlake.edu.cn, luwei@sutd.edu.sg Abstract We focus on graph-to-sequence learning, which can be framed as transducing graph structures to sequences for text generation. To capture structural information associated with graphs, we investigate the problem of encoding graphs using graph convolutional networks (GCNs). Unlike various existing approaches where shallow architectures were used for capturing local structural information only, we introduce a dense connection strategy, proposing a novel Densely Connected Graph Convolutional Network (DCGCN). Such a deep architecture is able to integrate both local and non-local features to learn a better structural representation of a graph. Our model outperforms the state-of-the-art neural models significantly on AMR-to-text generation and syntax-based neural machine translation. 1 Introduction Graphs play an important role in natural language processing (NLP) as they are able to capture richer structural information than sequences and trees. Generally, semantics of sentences can be encoded as graphs. For example, the abstract meaning representation (AMR) (Banarescu et al., 2013) is a directed, labeled graph as shown in Figure 1, where nodes in the graph denote semantic concepts and edges denote relations between concepts. Such graph representations can capture rich semantic- level structural information, and are attractive representations useful for semantics-related tasks such as semantic parsing (Guo and Lu, 2018) and natural language generation (Beck et al., 2018). In this paper, we focus on the graph-to-sequence ∗ Contributed equally. learning tasks, where we aim to learn repre- sentations for graphs that are useful for text generation. Graph convolutional networks (GCNs) (Kipf and Welling, 2017) are variants of convolutional neural networks (CNNs) that operate directly on graphs, where the representation of each node is iteratively updated based on those of its adja- cent nodes in the graph through an information propagation scheme. For example, the first layer of GCNs can only capture the graph’s adjacency information between immediate neighbors, while with the second layer one will be able to cap- ture second-order proximity information (neigh- borhood information two hops away from one node) as shown in Figure 1. Formally, L layers will be needed in order to capture neighborhood information that is L hops away. GCNs have been successfully applied to many NLP tasks (Bastings et al., 2017; Zhang et al., 2018b). Interestingly, although deeper GCNs with more layers will be able to capture richer neigh- borhood information of a graph, empirically it has been observed that the best performance is achieved with a 2-layer model (Li et al., 2018). Therefore, recent efforts that leverage recurrence- based graph neural networks have been explored as the alternatives to encode the structural infor- mation of graphs. Examples include graph-state long short-term memory (LSTM) networks (Song et al., 2018) and gated graph neural networks (GGNNs) (Beck et al., 2018). Deep architectures based on such recurrence-based models have been successfully built for tasks such as …

Content truncated to 500 words