Neural-Symbolic Inference for Robust Autoregressive Graph Parsing via Compositional Uncertainty Quantification

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4759 - 4776 December 7-11, 2022 ©2022 Association for Computational Linguistics Neural-Symbolic Inference for Robust Autoregressive Graph Parsing via Compositional Uncertainty Quantification Zi Lin UC San Diego lzi@ucsd.edu Jeremiah Liu†‡ Google Research & Harvard University jereliu@google.com Jingbo Shang† UC San Diego jshang@ucsd.edu Abstract Pre-trained seq2seq models excel at graph se- mantic parsing with rich annotated data, but generalize worse to out-of-distribution (OOD) and long-tail examples. In comparison, sym- bolic parsers under-perform on population- level metrics, but exhibit unique strength in OOD and tail generalization. In this work, we study compositionality-aware approach to neural-symbolic inference informed by model confidence, performing fine-grained neural- symbolic reasoning at subgraph level (i.e., nodes and edges) and precisely targeting sub- graph components with high uncertainty in the neural parser. As a result, the method combines the distinct strength of the neural and symbolic approaches in capturing different aspects of the graph prediction, leading to well-rounded gen- eralization performance both across domains and in the tail. We empirically investigate the approach in the English Resource Grammar (ERG) parsing problem on a diverse suite of standard in-domain and seven OOD corpora. Our approach leads to 35.26% and 35.60% er- ror reduction in aggregated SMATCH score over neural and symbolic approaches respectively, and 14% absolute accuracy gain in key tail lin- guistic categories over the neural model, out- performing prior state-of-art methods that do not account for compositionality or uncertainty. 1 Introduction A structured account of compositional meaning has become a longstanding goal for Natural Language Processing. To this end, a number of efforts have focused on encoding semantic relationships and at- tributes into graph-based meaning representations (MRs, see Appendix A for details). In particular, graph semantic parsing has been an important task in almost every Semantic Evaluation (SemEval) exercise since 2014. In recent years, we have wit- nessed the burgeoning of applying neural networks † Co-senior authors. ‡ Work done at Google. to semantic parsing. Pre-trained language model- based approaches have led to significant improve- ments across different MRs (Oepen et al., 2019, 2020). However, these models often generalize poorly to out-of-distribution (OOD) and tail ex- amples (Cheng et al., 2019; Shaw et al., 2021; Kim, 2021; Lin et al., 2022), while grammar or rule-based parser work relatively robustly across different linguistic phenomena and language do- mains (Cao et al., 2021; Lin et al., 2022). See Section 6 for a review of related work. In this paper, we propose a novel compositional neural-symbolic inference for graph semantic pars- ing, which takes advantage of both uncertainty quantification from a seq2seq parser and prior knowledge from a symbolic parser at the subgraph level (i.e., nodes and edges). We take graph seman- tic parsing for English Resource Grammar (ERG) as our case study. ERG is a compositional semantic representation explicitly coupled with the syntactic structure. Compared to other graph-based meaning representations like Abstract Meaning Representa- tion (AMR), ERG has high coverage of English text and strong transferability across …

Content truncated to 500 words