Graph-based Dependency Parsing with Graph Neural Networks

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2475–2485 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2475 Graph-based Dependency Parsing with Graph Neural Networks Tao Ji , Yuanbin Wu , and Man Lan Department of Computer Science and Technology, East China Normal University {taoji.cs}@gmail.com {ybwu,mlan}@cs.ecnu.edu.cn Abstract We investigate the problem of efficiently in- corporating high-order features into neural graph-based dependency parsing. Instead of explicitly extracting high-order features from intermediate parse trees, we develop a more powerful dependency tree node representation which captures high-order information con- cisely and efficiently. We use graph neu- ral networks (GNNs) to learn the representa- tions and discuss several new configurations of GNN’s updating and aggregation functions. Experiments on PTB show that our parser achieves the best UAS and LAS on PTB (96.0%, 94.3%) among systems without using any external resources. 1 Introduction In recent development of dependency parsers, learning representations is gaining in importance. From observed features (words, positions, POS tags) to latent parsing states, building expressive representations is shown to be crucial for getting accurate and robust parsing performances. Here we focus on graph-based dependency parsers. Given a sentence, a parser first scores all word pairs about how possible they hold valid de- pendency relations, and then use decoders (e.g., greedy, maximum spanning tree) to generate a full parse tree from the scores. The score function is a key component in graph-based parses. Com- monly, a neural network is assigned to learn low dimension vectors for words (i.e., nodes of parse trees), and the score function depends on vectors of the word pair (e.g., inner products). The main task of this paper is to explore effective encoding systems for dependency tree nodes. Two remarkable prior works on node repre- sentation are recurrent neural networks (RNNs) (Kiperwasser and Goldberg, 2016b) and biaffine mappings (Dozat and Manning, 2017). RNNs are powerful tools to collect sentence-level informa- tion, but the representations ignore features related to dependency structures. The biaffine mappings improve vanilla RNNs via a key observation: the representation of a word should be different re- garding whether it is a head or a dependent (i.e., dependency tree edges are directional). Therefore, Dozat and Manning (2017) suggest distinguishing head and dependent vector of a word. Following this line of thought, it is natural to ask whether we can introduce more structured knowledge into node representations. In other words, if biaffine mappings encode the first order parent-children re- lations, can we incorporate other high-order rela- tions (such as grandparents and siblings)? In this work, we propose to use graph neural networks (GNNs) for learning dependency tree node representations. Given a weighted graph, a GNN embeds a node by recursively aggregating node representations of its neighbours. For the parsing task, we build GNNs on weighted com- plete graphs which are readily obtained in graph- based parsers. The graphs could be fixed in prior or revised during the parsing process. By stack- ing multiple layers of GNNs, the representation …

Content truncated to 500 words