Comparing Knowledge-Intensive and Data-Intensive Models for English Resource Semantic Parsing

Comparing Knowledge-Intensive and Data-Intensive Models for English Resource Semantic Parsing Junjie Cao∗ Peking University Wangxuan Institute of Computer Technology junjie.junjiecao@alibaba-inc.com Zi Lin∗ Peking University Department of Chinese Language and Literature lzi@google.com Weiwei Sun∗∗ Peking University Wangxuan Institute of Computer Technology and Center for Chinese Linguistics ws390@cam.ac.uk Xiaojun Wan Peking University Wangxuan Institute of Computer Technology wanxiaojun@pku.edu.cn In this work, we present a phenomenon-oriented comparative analysis of the two dominant approaches in English Resource Semantic (ERS) parsing: classic, knowledge-intensive and neu- ral, data-intensive models. To reflect state-of-the-art neural NLP technologies, a factorization- based parser is introduced that can produce Elementary Dependency Structures much more ∗ Equal contribution. The work was done while the first three authors were at Peking University. Junjie Cao is now at Alibaba, and Zi Lin is now at Google. ∗∗ Corresponding author. Now at the Department of Computer Science and Technology of University of Cambridge. Submission received: 20 April 2019; revised version received: 3 October 2020; accepted for publication: 18 November 2020. https://doi.org/10.1162/COLI a 00395 © 2021 Association for Computational Linguistics Published under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license Computational Linguistics Volume 47, Number 1 accurately than previous data-driven parsers. We conduct a suite of tests for different linguistic phenomena to analyze the grammatical competence of different parsers, where we show that, despite comparable performance overall, knowledge- and data-intensive models produce different types of errors, in a way that can be explained by their theoretical properties. This analysis is beneficial to in-depth evaluation of several representative parsing techniques and leads to new directions for parser development. 1. Introduction Recent work in task-independent semantic parsing shifts from the knowledge-intensive approach to the data-intensive approach. Early attempts in semantic parsing leverage explicitly expressed symbolic rules in a deep grammar formalism, for example, Com- binatory Categorial Grammar (CCG; Steedman 1996, 2000) and Head-driven Phrase Structure Grammar (HPSG; Pollard and Sag 1994), to model the syntactico-semantic composition process (Bos et al. 2004; Callmeier 2000). Then, statistical machine learning technologies, especially structured prediction models, are utilized to enhance deep grammar-driven parsers (Clark and Curran 2007; Zhang, Oepen, and Carroll 2007; Miyao and Tsujii 2008). Recently, various deep learning models together with vector- based embeddings induced from large-scale raw texts have been making considerable advances (Chen, Sun, and Wan 2018; Dozat and Manning 2018). This article is concerned with comparing knowledge-intensive and data-intensive parsing models for English Resource Semantics (ERS; Flickinger, Bender, and Oepen 2014b,2014a), a comprehensive framework for in-depth linguistic analysis. Figures 1 and 2 are two examples illustrating the ERS representations. Our comparison is based not only on the general evaluation metrics for semantic parsing, but also a fine- grained construction-focused evaluation that sheds light on the kinds of strengths each type of parser exhibits. Characterizing such models may benefit parser development for not only ERS but also other frameworks, for example, Groningen Meaning Bank (GMB; Basile et al. 2012; Bos et al. 2017) and Abstract Meaning Representation (AMR; Banarescu et al. 2013). To reflect the state-of-the-art …

Content truncated to 500 words