Second-Order Semantic Dependency Parsing with End-to-End Neural Networks

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4609–4618 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4609 Second-Order Semantic Dependency Parsing with End-to-End Neural Networks Xinyu Wang, Jingxian Huang, Kewei Tu∗ School of Information Science and Technology, ShanghaiTech University, Shanghai, China {wangxy1,huangjx,tukw}@shanghaitech.edu.cn Abstract Semantic dependency parsing aims to iden- tify semantic relationships between words in a sentence that form a graph. In this paper, we propose a second-order semantic depen- dency parser, which takes into consideration not only individual dependency edges but also interactions between pairs of edges. We show that second-order parsing can be approximated using mean field (MF) variational inference or loopy belief propagation (LBP). We can unfold both algorithms as recurrent layers of a neural network and therefore can train the parser in an end-to-end manner. Our experi- ments show that our approach achieves state- of-the-art performance. 1 Introduction Semantic dependency parsing (Oepen et al.) aims to produce graph-structured semantic depen- dency representations of sentences instead of tree- structured syntactic dependency parses. Exist- ing approaches to semantic dependency parsing can be classified as graph-based approaches and transition-based approaches. In this paper, we investigate graph-based approaches which score each possible parse of a sentence by factorizing over its parts and search for the highest-scoring parse. Previous work in graph-based syntactic depen- dency parsing has shown that higher-order pars- ing generally outperforms first-order parsing (Mc- Donald and Pereira, 2006; Carreras, 2007; Koo and Collins, 2010; Ma and Zhao, 2012). While a first-order parser scores dependency edges in- dependently, a higher-order parser takes relation- ships between two or more edges into considera- tion. However, most of the previous algorithms for higher-order syntactic dependency tree pars- ing are not applicable to semantic dependency ∗Corresponding Author graph parsing, and designing efficient algorithms for higher-order semantic dependency graph pars- ing is nontrivial. In addition, it becomes a com- mon practice to use neural networks to compute features and scores of parse graph components, which ideally requires backpropagation of parsing errors through the higher-order parsing algorithm, adding to the difficulty of designing such an algo- rithm. In this paper, we propose a novel graph-based second-order semantic dependency parser. Given an input sentence, we use a neural network to com- pute scores for both first and second-order parts of parse graphs and then apply either mean field variational inference or loopy belief propagation to approximately find the highest-scoring parse graph. Both algorithms are iterative inference al- gorithms and we show that they can be unfolded as recurrent layers of a neural network with each layer representing the computation in one itera- tion of the algorithms. In this way, we can con- struct an end-to-end neural network that takes in a sentence and outputs the approximate marginal probability of every possible dependency edge. During training, we maximize the probability of the gold parses by using standard gradient-based methods. Our experiments show that our approach achieves state-of-the-art performance in semantic dependency parsing and outperforms our baseline …

Content truncated to 500 words