Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 187–197, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs Swabha Swayamdipta♣ Miguel Ballesteros♦ Chris Dyer♠ Noah A. Smith♥ ♣School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA ♦Natural Language Processing Group, Universitat Pompeu Fabra, Barcelona, Spain ♠Google DeepMind, London, UK ♥Computer Science & Engineering, University of Washington, Seattle, WA 98195, USA swabha@cs.cmu.edu, miguel.ballesteros@upf.edu, cdyer@cs.cmu.edu, nasmith@cs.washington.edu Abstract We present a transition-based parser that jointly produces syntactic and semantic dependencies. It learns a representation of the entire algorithm state, using stack long short-term memories. Our greedy in- ference algorithm has linear time, includ- ing feature extraction. On the CoNLL 2008–9 English shared tasks, we obtain the best published parsing performance among models that jointly learn syntax and semantics. 1 Introduction We introduce a new joint syntactic and semantic dependency parser. Our parser draws from the algorithmic insights of the incremental structure building approach of Henderson et al. (2008), with two key differences. First, it learns representations for the parser’s entire algorithmic state, not just the top items on the stack or the most recent parser states; in fact, it uses no expert-crafted features at all. Second, it uses entirely greedy inference rather than beam search. We find that it outperforms all previous joint parsing models, including Hender- son et al. (2008) and variants (Gesmundo et al., 2009; Titov et al., 2009; Henderson et al., 2013) on the CoNLL 2008 and 2009 (English) shared tasks. Our parser’s multilingual results are comparable to the top systems at CoNLL 2009. Joint models like ours have frequently been pro- posed as a way to avoid cascading errors in NLP pipelines; varying degrees of success have been at- tained for a range of joint syntactic-semantic anal- ysis tasks (Sutton and McCallum, 2005; Hender- son et al., 2008; Toutanova et al., 2008; Johansson, 2009; Llu´ıs et al., 2013, inter alia). One reason pipelines often dominate is that they make available the complete syntactic parse tree, and arbitrarily-scoped syntactic features—such as the “path” between predicate and argument, pro- posed by Gildea and Jurafsky (2002)—for seman- tic analysis. Such features are a mainstay of high- performance semantic role labeling (SRL) sys- tems (Roth and Woodsend, 2014; Lei et al., 2015; FitzGerald et al., 2015; Foland and Martin, 2015), but they are expensive to extract (Johansson, 2009; He et al., 2013). This study shows how recent advances in repre- sentation learning can bypass those expensive fea- tures, discovering cheap alternatives available dur- ing a greedy parsing procedure. The specific ad- vance we employ is the stack LSTM (Dyer et al., 2015), a neural network that continuously summa- rizes the contents of the stack data structures in which a transition-based parser’s state is conven- tionally encoded. Stack LSTMs were shown to ob- viate many features used in syntactic dependency parsing; here we find them to do the same for joint syntactic-semantic dependency parsing. We believe this is an …

Content truncated to 500 words