Semantic Role Labeling with Iterative Structure Refinement

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1071–1082, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1071 Semantic Role Labeling with Iterative Structure Refinement Chunchuan Lyu1 Shay B. Cohen1 Ivan Titov1,2 1ILCC, School of Informatics, University of Edinburgh 2ILLC, University of Amsterdam chunchuan.lv@gmail.com scohen@inf.ed.ac.uk ititov@inf.ed.ac.uk Abstract Modern state-of-the-art Semantic Role Label- ing (SRL) methods rely on expressive sentence encoders (e.g., multi-layer LSTMs) but tend to model only local (if any) interactions be- tween individual argument labeling decisions. This contrasts with earlier work and also with the intuition that the labels of individual argu- ments are strongly interdependent. We model interactions between argument labeling de- cisions through iterative refinement. Start- ing with an output produced by a factorized model, we iteratively refine it using a refine- ment network. Instead of modeling arbitrary interactions among roles and words, we en- code prior knowledge about the SRL problem by designing a restricted network architecture capturing non-local interactions. This model- ing choice prevents overfitting and results in an effective model, outperforming strong fac- torized baseline models on all 7 CoNLL-2009 languages, and achieving state-of-the-art re- sults on 5 of them, including English. 1 Introduction Semantic role labeling (SRL), originally intro- duced by Gildea and Jurafsky (2000), involves the prediction of predicate-argument structure, i.e., identification of arguments and their assignment to underlying semantic roles. Semantic-role rep- resentations have been shown to be beneficial in many NLP applications, including question an- swering (Shen and Lapata, 2007), information ex- traction (Christensen et al., 2011) and machine translation (Marcheggiani et al., 2018). In this work, we focus on dependency-based SRL (Hajiˇc et al., 2009), a popular version of the task which involves identifying syntactic heads of arguments rather than marking entire argument spans (see the graph in red in Figure 1). Edges in the depen- dency graphs are annotated with semantic roles (e.g., A0:PLEASER) and the predicates are labeled An standard for a state satisfy An easier standard for a state to entity fulfilled (A1) satisfy easier to pleaser (A0) entity fulfilled (A1) satisfy.01 satisfy.01 Initial Refined = gold An easier standard for a state to satisfy ... Figure 1: An example of structured refinement, the sen- tence fragment is from CoNLL-2009: the initial pre- diction by the factorized model in blue, the refined one (identical to the gold standard) in red. with their senses from a given sense inventory (e.g., SATISFY.01 in the example). Before the rise of deep learning methods, the most accurate SRL methods relied on model- ing high-order interactions in the output space (e.g., between arguments or arguments and pred- icates) (Watanabe et al., 2010; Toutanova et al., 2008). Earlier neural methods can model such output interactions through a transition system, and achieve competitive performance (Hender- son et al., 2013). However, current state-of- the-art SRL systems use powerful sentence en- coders (e.g., layers of LSTMs (Li et al., 2018; He et al., 2017) or multi-head …

Content truncated to 500 words