Deep Multitask Learning for Semantic Dependency Parsing

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 2037–2048 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1186 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 2037–2048 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1186 Deep Multitask Learning for Semantic Dependency Parsing Hao Peng∗ Sam Thomson† Noah A. Smith∗ ∗Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA †School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA {hapeng,nasmith}@cs.washington.edu, sthomson@cs.cmu.edu Abstract We present a deep neural architecture that parses sentences into three semantic de- pendency graph formalisms. By using ef- ficient, nearly arc-factored inference and a bidirectional-LSTM composed with a multi-layer perceptron, our base system is able to significantly improve the state of the art for semantic dependency parsing, without using hand-engineered features or syntax. We then explore two multitask learning approaches—one that shares pa- rameters across formalisms, and one that uses higher-order structures to predict the graphs jointly. We find that both ap- proaches improve performance across for- malisms on average, achieving a new state of the art. Our code is open-source and available at https://github.com/ Noahs-ARK/NeurboParser. 1 Introduction Labeled directed graphs are a natural and flexi- ble representation for semantics (Copestake et al., 2005; Baker et al., 2007; Surdeanu et al., 2008; Banarescu et al., 2013, inter alia). Their generality over trees, for instance, allows them to represent relational semantics while handling phenomena like coreference and coordination. Even syntactic formalisms are moving toward graphs (de Marn- effe et al., 2014). However, full semantic graphs can be expensive to annotate, and efforts are frag- mented across competing semantic theories, lead- ing to a limited number of annotations in any one formalism. This makes learning to parse more dif- ficult, especially for powerful but data-hungry ma- chine learning techniques like neural networks. In this work, we hypothesize that the overlap among theories and their corresponding represen- Last week , shareholders took their money and ran . arg1 loc top arg1 poss arg2 _and_c arg1 (a) DM arg1 top arg1 arg1 arg2 coord arg1 arg1 coord Last week , shareholders took their money ran . and (b) PAS Last week , shareholders took their money and ran . rstr twhen top act app pat conj act top twhen conj (c) PSD Figure 1: An example sentence annotated with the three semantic formalisms of the broad-coverage semantic dependency parsing shared tasks. tations can be exploited using multitask learn- ing (Caruana, 1997), allowing us to learn from more data. We use the 2015 SemEval shared task on Broad-Coverage Semantic Dependency Pars- ing (SDP; Oepen et al., 2015) as our testbed. The shared task provides an English-language cor- pus with parallel annotations for three semantic graph representations, described in §2. Though the shared task was designed in part to encourage comparison between the formalisms, we are the first to treat SDP as a multitask learning problem. As …

Content truncated to 500 words