Two Local Models for Neural Constituent Parsing

Proceedings of the 27th International Conference on Computational Linguistics, pages 119–132 Santa Fe, New Mexico, USA, August 20-26, 2018. 119 Two Local Models for Neural Constituent Parsing Zhiyang Teng and Yue Zhang Singapore University of Technology and Design zhiyang teng@mymail.sutd.edu.sg yue zhang@sutd.edu.sg Non-local features have been exploited by syntactic parsers for capturing dependencies between sub output structures. Such features have been a key to the success of state-of-the-art statistical parsers. With the rise of deep learning, however, it has been shown that local output decisions can give highly compet- itive accuracies, thanks to the power of dense neural input representations that embody global syntactic information. We investigate two conceptually simple local neural models for constituent parsing, which make local decisions to constituent spans and CFG rules, respectively. Consistent with previous find- ings along the line, our best model gives highly competitive results, achieving the labeled bracketing F1 scores of 92.4% on PTB and 87.3% on CTB 5.1. 1 Introduction Non-local features have been shown crucial for statistical parsing (Huang, 2008a; Zhang and Nivre, 2011). For dependency parsing, High-order dynamic programs (Koo and Collins, 2010), integer linear programming (Martins et al., 2010) and dual decomposition (Koo et al., 2010) techniques have been exploited by graph-based parser to integrate non-local features. Transition-based parsers (Nivre, 2003; Nivre, 2008; Zhang and Nivre, 2011; Bohnet, 2010; Huang et al., 2012) are also known for leveraging non-local features for achieving high accuracies. For most state-of-the-art statistical parsers, a global training objective over the entire parse tree has been defined to avoid label bias (Lafferty et al., 2001). For neural parsing, on the other hand, local models have been shown to give highly competitive ac- curacies (Cross and Huang, 2016b; Stern et al., 2017) as compared to those that employ long-range features (Watanabe and Sumita, 2015; Zhou et al., 2015; Andor et al., 2016; Durrett and Klein, 2015). Highly local features have been used in recent state-of-the-art models (Stern et al., 2017; Dozat and Manning, 2016; Shi et al., 2017). In particular, Dozat and Manning (2016) show that a locally trained arc-factored model can give the best reported accuracies on dependency parsing. The surprising result has been largely attributed to the representation power of long short-term memory (LSTM) encoders (Kiperwasser and Goldberg, 2016). An interesting research question is to what extent the encoding power can be leveraged for constituent parsing. We investigate the problem by building a chart-based model that is local to unlabeled constituent spans (Abney, 1991) and CFG-rules, which have been explored by early PCFG models (Collins, 2003; Klein and Manning, 2003). In particular, our models first predict unlabeled CFG trees leveraging bi- affine modelling (Dozat and Manning, 2016). Then, constituent labels are assigned on unlabeled trees by using a tree-LSTM to encode the syntactic structure, and a LSTM decoder for yielding label sequences on each node, which can include unary rules. Experiments show that our conceptually simple models give highly competitive performances com- pared with the state-of-the-art. Our best models give labeled bracketing F1 …

Content truncated to 500 words