Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference

Findings of the Association for Computational Linguistics: ACL 2022, pages 112 - 127 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Challenges to Open-Domain Constituency Parsing Sen Yang♠, Leyang Cui♡♠, Ruoxi Ning♡, Di Wu△, Yue Zhang♠3∗ ♠School of Engineering, Westlake University ♡Zhejiang University △Sichuan University 3Institute of Advanced Technology, Westlake Institute for Advanced Study senyang.stu@gmail.com {cuileyang, zhangyue}@westlake.edu.cn lune_rgb@163.com wuuuudle@gmail.com Abstract Neural constituency parsers have reached prac- tical performance on news-domain benchmarks. However, their generalization ability to other domains remains weak. Existing findings on cross-domain constituency parsing are only made on a limited number of domains. Track- ing this, we manually annotate a high-quality constituency treebank containing five domains. We analyze challenges to open-domain con- stituency parsing using a set of linguistic fea- tures on various strong constituency parsers. Primarily, we find that 1) BERT significantly in- creases parsers’ cross-domain performance by reducing their sensitivity on the domain-variant features. 2) Compared with single metrics such as unigram distribution and OOV rate, chal- lenges to open-domain constituency parsing arise from combinations of factors, including cross-domain lexical and constituent structure variations. 1 Introduction Constituency parsing is a fundamental task in NLP that has received constant research attention (Cross and Huang, 2016; Liu and Zhang, 2017; Stern et al., 2017; Kitaev and Klein, 2018). As shown in Figure 1, given a sentence, the task is to iden- tify hierarchical phrase structures that reflect its syntax, such as prepositional phrases (PP; e.g., “in late 1991”), noun phrases (NP; e.g., “late 1991”) and verb phrases (VP; e.g., “scheduled for deliv- ery in late 1991”). Constituent structures have been shown useful for downstream tasks including machine translation (Wang et al., 2018), natural language inference (Chen et al., 2017), text sum- marization (Xu and Durrett, 2019). In addition, they can be transformed into dependency tree struc- tures (Zhang and Clark, 2008), which have been shown to be useful for a wide range of NLP tasks. The dominant approach to constituency parsing employs a neural model with pre-trained token rep- ∗ Corresponding author. Figure 1: An example of constituency parse tree. resentation (Kitaev et al., 2019), training the net- work parameters over manually labeled constituent structures from the Penn Treebank (PTB) (Marcus et al., 1993). As labeled constituent trees can be costly to obtain, most work makes use of the PTB data for training, which is financial news. The cur- rent state-of-the-art F-scores reach over 95% on the training domain (i.e., newswire) and are around 88% for biomedical and web test data (Tateisi et al., 2005; Silveira et al., 2014). Compared with parser performance decades ago, accuracies around 90% nowadays is much more useful for downstream applications. Fried et al. (2019) showed that pre- training is a key factor that brings consistent cross- domain performance improvements by using BERT (Devlin et al., 2019). Ideally, a constituency parser should give ro- bust performance in the open domain, so that both domain-specific applications (Zhang et al., 2021) and open-domain NLP tasks (Hu et al., 2019) can benefit from syntactic structures. The …

Content truncated to 500 words