Parsing All: Syntax and Semantics, Dependencies and Spans

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4438–4449 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4438 Parsing All: Syntax and Semantics, Dependencies and Spans Junru Zhou1,2,3 , Zuchao Li 1,2,3, Hai Zhao1,2,3∗ 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China 3MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University {zhoujunru,charlee}@sjtu.edu.cn, zhaohai@cs.sjtu.edu.cn Abstract Both syntactic and semantic structures are key linguistic contextual clues, in which parsing the latter has been well shown beneficial from parsing the former. However, few works ever made an attempt to let semantic parsing help syntactic parsing. As linguistic representation formalisms, both syntax and semantics may be represented in either span (constituent/phrase) or dependency, on both of which joint learning was also seldom explored. In this paper, we propose a novel joint model of syntactic and semantic parsing on both span and dependency representations, which incorporates syntactic information effectively in the encoder of neu- ral network and benefits from two representa- tion formalisms in a uniform way. The experi- ments show that semantics and syntax can ben- efit each other by optimizing joint objectives. Our single model achieves new state-of-the-art or competitive results on both span and depen- dency semantic parsing on Propbank bench- marks and both dependency and constituent syntactic parsing on Penn Treebank. 1 Introduction This work makes the first attempt to fill the gaps on syntactic and semantic parsing from jointly consid- ering its representation forms and their linguistic processing layers. First, both span (constituent) and dependency are effective formal representations for both semantics and syntax, which have been well studied and discussed from both linguistic and computational perspective, though few works com- prehensively considered the impact of either/both representation styles over the respective parsing ∗ Corresponding author. This paper was partially sup- ported by National Key Research and Development Program of China (No. 2017YFB0304100), Key Projects of Na- tional Natural Science Foundation of China (U1836222 and 61733011), Huawei-SJTU long term AI project, Cutting-edge Machine reading comprehension and language model. (Chomsky, 1981; Li et al., 2019b). Second, as se- mantics is usually considered as a higher layer of linguistics over syntax, most previous studies focus on how the latter helps the former. Though there comes a trend that syntactic clues show less impact on enhancing semantic parsing since neural models were introduced (Marcheggiani and Titov, 2017). In fact, recent works (He et al., 2017; Marcheggiani et al., 2017) propose syntax-agnostic models for semantic parsing and achieve competitive and even state-of-the-art results. However, semantics may not only benefit from syntax which has been well known, but syntax may also benefit from seman- tics, which is an obvious gap in explicit linguistic structure parsing and few attempts were ever re- ported. To our best knowledge, few previous works focus on the relationship between syntax and se- mantic which only based on dependency structure (Swayamdipta et al., 2016; …

Content truncated to 500 words