Investigating Non-local Features for Neural Constituency Parsing

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2065 - 2075 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Investigating Non-local Features for Neural Constituency Parsing Leyang Cui♥♠∗, Sen Yang♠∗, Yue Zhang♠3† ♥Zhejiang University ♠School of Engineering, Westlake University 3Institute of Advanced Technology, Westlake Institute for Advanced Study {cuileyang, zhangyue}@westlake.edu.cn senyang.stu@gmail.com Abstract Thanks to the strong representation power of neural encoders, neural chart-based parsers have achieved highly competitive performance by using local features. Recently, it has been shown that non-local features in CRF struc- tures lead to improvements. In this paper, we investigate injecting non-local features into the training process of a local span-based parser, by predicting constituent n-gram non-local patterns and ensuring consistency between non-local patterns and local constituents. Re- sults show that our simple method gives bet- ter results than the self-attentive parser on both PTB and CTB. Besides, our method achieves state-of-the-art BERT-based performance on PTB (95.92 F1) and strong performance on CTB (92.31 F1). Our parser also achieves bet- ter or competitive performance in multilingual and zero-shot cross-domain settings compared with the baseline. 1 Introduction Constituency parsing is a fundamental task in nat- ural language processing, which provides useful information for downstream tasks such as machine translation (Wang et al., 2018), natural language in- ference (Chen et al., 2017), text summarization (Xu and Durrett, 2019). Over the recent years, with advance in deep learning and pre-training, neu- ral chart-based constituency parsers (Stern et al., 2017a; Kitaev and Klein, 2018) have achieved highly competitive results on benchmarks like Penn Treebank (PTB) and Penn Chinese Treebank (CTB) by solely using local span prediction. The above methods take the contextualized rep- resentation (e.g., BERT) of a text span as input, and use a local classifier network to calculate the scores of the span being a syntactic constituent, together with its constituent label. For testing, the output layer uses a non-parametric dynamic programming ∗ The first two authors contributed equally to this work. † Corresponding author. Figure 1: An example of the non-local n-gram pat- tern features: the 3-gram pattern (3, 11, {VBD NP PP}) is composed of two constituent nodes and one part- of-speech node; the 2-gram pattern (7, 11, {NP PP}) is composed of two constituent nodes. algorithm (e.g., CKY) to find the highest-scoring tree. Without explicitly modeling structured depen- dencies between different constituents, the methods give competitive results compared to non-local dis- crete parsers (Stern et al., 2017a; Kitaev and Klein, 2018). One possible explanation for their strong performance is that the powerful neural encoders are capable of capturing implicit output correlation of the tree structure (Stern et al., 2017a; Gaddy et al., 2018; Teng and Zhang, 2018). Recent work has shown that modeling non-local output dependencies can benefit neural structured prediction tasks, such as NER (Ma and Hovy, 2016), CCG supertagging (Cui and Zhang, 2019) and dependency parsing (Zhang et al., 2020a). Thus, an interesting research question is whether injecting non-local tree structure features is also beneficial …

Content truncated to 500 words