A Minimal Span-Based Neural Constituency Parser

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 818–827 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1076 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 818–827 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1076 A Minimal Span-Based Neural Constituency Parser Mitchell Stern Jacob Andreas Dan Klein Computer Science Division University of California, Berkeley {mitchell,jda,klein}@cs.berkeley.edu Abstract In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans. We show that this model is not only compatible with classical dynamic pro- gramming techniques, but also admits a novel greedy top-down inference algo- rithm based on recursive partitioning of the input. We demonstrate empirically that both prediction schemes are competi- tive with recent work, and when combined with basic extensions to the scoring model are capable of achieving state-of-the-art single-model performance on the Penn Treebank (91.79 F1) and strong perfor- mance on the French Treebank (82.23 F1). 1 Introduction This paper presents a minimal but surprisingly ef- fective span-based neural model for constituency parsing. Recent years have seen a great deal of interest in parsing architectures that make use of recurrent neural network (RNN) representations of input sentences (Vinyals et al., 2015). Despite evidence that linear RNN decoders are implicitly able to respect some nontrivial well-formedness constraints on structured outputs (Graves, 2013), researchers have consistently found that the best performance is achieved by systems that explicitly require the decoder to generate well-formed tree structures (Chen and Manning, 2014). There are two general approaches to ensuring this structural consistency. The most common is to encode the output as a sequence of operations within a transition system which constructs trees incrementally. This transforms the parsing prob- lem back into a sequence-to-sequence problem, while making it easy to force the decoder to take only actions guaranteed to produce well-formed outputs. However, transition-based models do not admit fast dynamic programs and require careful feature engineering to support exact search-based inference (Thang et al., 2015). Moreover, models with recurrent state require complex training pro- cedures to benefit from anything other than greedy decoding (Wiseman and Rush, 2016). An alternative line of work focuses on chart parsers, which use log-linear or neural scoring potentials to parameterize a tree-structured dy- namic program for maximization or marginal- ization (Finkel et al., 2008; Durrett and Klein, 2015). These models enjoy a number of appeal- ing formal properties, including support for ex- act inference and structured loss functions. How- ever, previous chart-based approaches have re- quired considerable scaffolding beyond a simple well-formedness potential, e.g. pre-specification of a complete context-free grammar for generat- ing output structures and initial pruning of the out- put space with a weaker model (Hall et al., 2014). Additionally, we are unaware of any recent chart- based models that achieve results competitive with the best transition-based models. In this work, we present an extremely simple chart-based neural parser based …

Content truncated to 500 words