What’s Going On in Neural Constituency Parsers? An Analysis

Proceedings of NAACL-HLT 2018, pages 999–1010 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics What’s Going On in Neural Constituency Parsers? An Analysis David Gaddy, Mitchell Stern, and Dan Klein Computer Science Division University of California, Berkeley {dgaddy,mitchell,klein}@berkeley.edu Abstract A number of differences have emerged be- tween modern and classic approaches to con- stituency parsing in recent years, with struc- tural components like grammars and feature- rich lexicons becoming less central while re- current neural network representations rise in popularity. The goal of this work is to ana- lyze the extent to which information provided directly by the model structure in classical sys- tems is still being captured by neural methods. To this end, we propose a high-performance neural model (92.08 F1 on PTB) that is rep- resentative of recent work and perform a se- ries of investigative experiments. We find that our model implicitly learns to encode much of the same information that was explicitly pro- vided by grammars and lexicons in the past, indicating that this scaffolding can largely be subsumed by powerful general-purpose neural machinery. 1 Introduction In the past several years, many aspects of con- stituency parsing and natural language processing in general have changed. Grammars, which were once the central component of many parsers, have played a continually decreasing role. Rich lexi- cons and handcrafted lexical features have become less common as well. On the other hand, recurrent neural networks have gained traction as a power- ful and general purpose tool for representation. So far, not much has been shown about how neural networks are able to compensate for the removal of the structures used in past models. To gain in- sight, we introduce a parser that is representative of recent trends and analyze its learned represen- tations to determine what information it captures and what is important for its strong performance. Our parser is a natural extension of recent work in constituency parsing. We combine a common span representation based on recurrent neural net- works with a novel, simplified scoring model. In addition, we replace the externally predicted part- of-speech tags used in some recent systems with character-level word representations. Our parser achieves a test F1 score of 92.08 on section 23 of the Penn Treebank, exceeding the performance of many other state-of-the-art models evaluated un- der comparable conditions. Section 2 describes our model in detail. The remainder of the paper is focused on anal- ysis. In Section 3, we look at the decline of grammars and output correlations. Past work in constituency parsing used context-free grammars with production rules governing adjacent labels (or more generally production-factored scores) to propagate information and capture correlations be- tween output decisions (Collins, 1997; Charniak and Johnson, 2005; Petrov and Klein, 2007; Hall et al., 2014). Many recent parsers no longer have explicit grammar production rules, but still use in- formation about other predictions, allowing them to capture output correlations (Dyer et al., 2016; Choe and Charniak, 2016). Beyond this, there are some …

Content truncated to 500 words