Syntax-Enhanced Self-Attention-Based Semantic Role Labeling

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 616–626, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 616 Syntax-Enhanced Self-Attention-Based Semantic Role Labeling Yue Zhang, Rui Wang, Luo Si Alibaba Group, China {shiyu.zy, masi.wr, luo.si}@alibaba-inc.com Abstract As a fundamental NLP task, semantic role la- beling (SRL) aims to discover the semantic roles for each predicate within one sentence. This paper investigates how to incorporate syntactic knowledge into the SRL task effec- tively. We present different approaches of en- coding the syntactic information derived from dependency trees of different quality and rep- resentations; we propose a syntax-enhanced self-attention model and compare it with other two strong baseline methods; and we con- duct experiments with newly published deep contextualized word representations as well. The experiment results demonstrate that with proper incorporation of the high quality syn- tactic information, our model achieves a new state-of-the-art performance for the Chinese SRL task on the CoNLL-2009 dataset. 1 Introduction The task of semantic role labeling (SRL) is to rec- ognize arguments for a given predicate in one sen- tence and assign labels to them, including “who” did “what” to “whom”, “when”, “where”, etc. Fig- ure 1 is an example sentence with both semantic roles and syntactic dependencies. Since the nature of semantic roles is more abstract than the syntac- tic dependencies, SRL has a wide range of appli- cations in different areas, e.g., text classification (Sinoara et al., 2016), text summarization (Gen- est and Lapalme, 2011; Khan et al., 2015), recog- nizing textual entailment (Burchardt et al., 2007; Stern and Dagan, 2014), information extraction (Surdeanu et al., 2003), question answering (Shen and Lapata, 2007; Yih et al., 2016), and so on. Traditionally, syntax is the bridge to reach se- mantics. However, along with the popularity of the end-to-end models in the NLP community, various recent studies have been discussing the necessity of syntax in the context of SRL. For instance, He $ 中国 鼓励 外商 投资 农业 $ China encourage foreign invest agriculture merchant SBJ ROOT COMP COMP COMP A0 A1 A2 Figure 1: An example of one sentence with its syntac- tic dependency tree and semantic roles. Arcs above the sentence are semantic role annotations for the predicate “鼓励 (encourage)” and below the sentence are syntac- tic dependency annotations of the whole sentence. The meaning of this sentence is “China encourages foreign merchants to invest in agriculture”. et al. (2017) have observed that only good syntax helps with the SRL performance. Xia et al. (2019) have explored what kind of syntactic information or structure is better suited for the SRL model. Cai et al. (2018) have compared syntax-agnostic and syntax-aware approaches and claim that the syntax-agnostic model surpasses the syntax-aware ones. In this paper, we focus on analyzing the rela- tionship between the syntactic dependency infor- mation and the SRL performance. In particular, we investigate the following four aspects: 1) Qual- ity of the syntactic information: …

Content truncated to 500 words