Neural Combinatory Constituency Parsing

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2199–2213 August 1–6, 2021. ©2021 Association for Computational Linguistics 2199 Neural Combinatory Constituency Parsing Zhousi Chen, Longtu Zhang, Aizhan Imankulova, and Mamoru Komachi Tokyo Metropolitan University 6-6 Asahigaoka, Hino, Tokyo 191-0065, Japan chen-zhousi@ed.tmu.ac.jp vincentzlt@outlook.com aizhan.imankulova@cogsmart-global.com komachi@tmu.ac.jp Abstract We propose two fast neural combinatory mod- els for constituency parsing: binary and multi- branching. Our models decompose the bottom- up parsing process into 1) classification of tags, labels, and binary orientations or chunks and 2) vector composition based on the computed orientations or chunks. These models have theoretical sub-quadratic complexity and em- pirical linear complexity. The binary model achieves an F1 score of 92.54 on Penn Tree- bank, speeding at 1327.2 sents/sec. Both the models with XLNet provide near state-of-the- art accuracies for English. Syntactic branching tendency and headedness of a language are ob- served during the training and inference pro- cesses for Penn Treebank, Chinese Treebank, and Keyaki Treebank (Japanese). 1 Introduction Transition-based and chart-based methods are two main paradigms for constituency parsing. Transition-based parsers (Dyer et al., 2016; Kitaev and Klein, 2020) build a tree with a sequence of local actions. Despite their O(n) computational complexity, the locality makes them less accurate and necessitates additional grammars or lookahead features for improvement (Kuhlmann et al., 2011; Zhu et al., 2013; Liu and Zhang, 2017c). By con- trast, chart-based parsers are conceptually simple and accurate when used with a CYK-style algo- rithm (Kitaev and Klein, 2018; Zhou and Zhao, 2019) for finding the global optima. However, their complexity is O(n3). To achieve both accuracy and simplicity (without high complexity) is a criti- cal problem in parsing. Recent efforts were made using neural models. In contrast to earlier symbolic approaches (Char- niak, 2000; Klein and Manning, 2003), neural mod- els are simplified by utilizing their adaptive dis- tributed representation, thereby eliminating compli- Yet parents demand them . CC NNS VBP PRP . #CC NP #VBP NP #. _S VP #. _S #. S R R R R L L L L L L BiLSTMori FFNN ori FFNN label FFNN tag FFNN binary Model Components: L/R BiLSTM cxt or XLNet Figure 1: Parsing instance with the binary model. The bottom-up flow of word information is indicated by blue arrows and orientation flows by dotted red arrows. Binary parsing explores the internal constituents of S. Special labels prefixed with “#” or “ ” are sub category placeholders caused by binarization and stratification. cated symbolic engineering. The seq2seq model for parsing (Vinyals et al., 2015) leverages such repre- sentation to interpret the structural task as a general sequential task. With augmented data and ensem- ble, it outperforms the symbolic models mentioned in Petrov et al. (2006) and provides a complexity of O(n2) with the attention mechanism (Bahdanau et al., 2015). However, its performance is infe- rior to those of specialized neural parsers (Liu and Zhang, 2017a,b,c). Socher et al. (2013) proposed a parsing strategy for a symbolic constituent parser augmented with neural vector compositionality. It did …

Content truncated to 500 words