Joint Universal Syntactic and Semantic Parsing

Joint Universal Syntactic and Semantic Parsing Elias Stengel-Eskin Johns Hopkins University, United States Kenton Murray Johns Hopkins University, United States Sheng Zhang Microsoft Research, United States Aaron Steven White University of Rochester, United States Benjamin Van Durme Johns Hopkins University, United States Abstract While numerous attempts have been made to jointly parse syntax and semantics, high performance in one domain typically comes at the price of performance in the other. This trade-off contradicts the large body of re- search focusing on the rich interactions at the syntax–semantics interface. We explore multi- ple model architectures that allow us to exploit the rich syntactic and semantic annotations contained in the Universal Decompositional Semantics (UDS) dataset, jointly parsing Universal Dependencies and UDS to obtain state-of-the-art results in both formalisms. We analyze the behavior of a joint model of syn- tax and semantics, finding patterns supported by linguistic theory at the syntax–semantics interface. We then investigate to what degree joint modeling generalizes to a multilingual setting, where we find similar trends across 8 languages. 1 Introduction Given their natural expression in terms of hier- archical structures and their well-studied interac- tions, syntax and semantics have long been treated as parsing tasks, both independently and jointly. One would expect joint models to outperform sep- arate or pipelined ones; however, many previous attempts have yielded mixed results, finding that while one level can be used as an additional signal to benefit the other (Swayamdipta et al., 2017, 2018; Johansson and Nugues, 2008), obtaining high performance in both syntax and semantics simultaneously is difficult (Krishnamurthy and Mitchell, 2014; Hajiˇc et al., 2009). A variety of tree- and graph-based representa- tions have been devised for representing syntactic structure (e.g., varieties of constituency and dependency parse trees) as well as semantic struc- ture, for example, Abstract Meaning Represen- tation (AMR; Banarescu et al., 2013), Universal Conceptual Cognitive Annotation (UCCA; Abend and Rappoport, 2013), and Semantic Dependency Parsing formalisms (SDP; Oepen et al., 2014, 2016). These semantic representations have vary- ing degrees of abstraction from the input and syntax, ranging from being directly tied to the input tokens (e.g., SDP formalisms) to being heavily abstracted away from it (e.g., AMR, UCCA). Universal Decompositional Semantics (UDS; White et al., 2020) falls between these extremes, with a semantic graph that is closely tied to the syntax while not being constrained to match the input tokens. Crucially, UDS graphs not only represent the predicate-argument rela- tionships in the input, but also host scalar-valued crowdsourced annotations encoding a variety of semantic inferences, described in §3. These pro- vide another level for linguistic analysis and make UDS unique among meaning representations. Furthermore, as UDS graphs build on Universal Dependency (UD) parses, UDS is naturally posi- tioned to take full advantage of the extensive and linguistically diverse set of UD annotations. We extend the transductive sequence-to-graph UDS parser proposed by Stengel-Eskin et al. (2020) to simultaneously perform UD and UDS parsing, finding that joint modeling offers con- comitant benefits to both tasks. In particular, after exploring …

Content truncated to 500 words