Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty

Findings of the Association for Computational Linguistics: ACL 2022, pages 4160 - 4173 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty Zi Lin ∗ UC San Diego lzi@ucsd.edu Jeremiah Liu Google Research & Harvard University jereliu@google.com Jingbo Shang UC San Diego jshang@ucsd.edu Abstract Recent work in task-independent graph seman- tic parsing has shifted from grammar-based symbolic approaches to neural models, show- ing strong performance on different types of meaning representations. However, it is still unclear that what are the limitations of these neural parsers, and whether these limitations can be compensated by incorporating symbolic knowledge into model inference. In this pa- per, we address these questions by taking En- glish Resource Grammar (ERG) parsing as a case study. Specifically, we first develop a state-of-the-art, T5-based neural ERG parser, and conduct detail analyses of parser perfor- mance within fine-grained linguistic categories. The neural parser attains superior performance on in-distribution test set, but degrades signif- icantly on long-tail situations, while the sym- bolic parser performs more robustly. To address this, we further propose a simple yet principled collaborative framework for neural-symbolic semantic parsing, by designing a decision cri- terion for beam search that incorporates the prior knowledge from a symbolic parser and accounts for model uncertainty. Experimen- tal results show that the proposed framework yields comprehensive improvement over neural baseline across long-tail categories, yielding the best known SMATCH score (97.01) on the well-studied DeepBank benchmark. 1 Introduction Semantic parsing is the task of mapping natural language to machine interpretable meaning repre- sentations, and graph-structured semantic represen- tations, which encode rich semantic information in the form of semantic graphs, have played an im- portant role in natural language processing (Oepen et al., 2019). Parsing natural language sentences into the semantic-graph representation (e.g., Figure 1) has ∗Part of the work was done while Zi was an AI resident at Google. been extensively studied in the recent decade. Work in this area has shifted from the symbolic (grammar-based) approach to the neural approach. Thanks to the flourishing of deep learning technolo- gies, sequence-to-sequence (seq2seq) models have shown great performance on data sampled from the training distribution. These neural semantic parsers reduce the need for domain-specific gram- mar and feature engineering, but comes at a cost of lacking interpretability, as the model directly out- puts a (linearized) graph without revealing the un- derlying meaning-composition process. Moreover, these neural models often generalize poorly to tail and out-of-distribution (OOD) examples, and previ- ous work has shown that combining high-precision symbolic approaches with neural models can ad- dress this issue for task-oriented semantic parsing (Shaw et al., 2021; Kim, 2021; Cheng et al., 2019). However, this type of approach requires complex architecture engineering to incorporate the gram- mar formalism. The grammar formalism being utilized is usually primitive, and was not tested beyond simple datasets such as SCAN (Lake and Baroni, 2018) or GEOQUERY (Zelle and Mooney, 1996). Therefore they are likely not sufficient for handling complex graph-based meaning represen- tations derived …

Content truncated to 500 words