Capturing Argument Interaction in Semantic Role Labeling with Capsule Networks

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5415–5425, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5415 Capturing Argument Interaction in Semantic Role Labeling with Capsule Networks Xinchi Chen1 Chunchuan Lyu1 Ivan Titov1,2 xchen13@exseed.ed.ac.uk chunchuan.lv@gmail.com ititov@inf.ed.ac.uk 1 ILCC, School of Informatics, University of Edinburgh 2ILLC, University of Amsterdam Abstract Semantic role labeling (SRL) involves extract- ing propositions (i.e. predicates and their typed arguments) from natural language sen- tences. State-of-the-art SRL models rely on powerful encoders (e.g., LSTMs) and do not model non-local interaction between argu- ments. We propose a new approach to mod- eling these interactions while maintaining ef- ficient inference. Specifically, we use Capsule Networks (Sabour et al., 2017): each propo- sition is encoded as a tuple of capsules, one capsule per argument type (i.e. role). These tuples serve as embeddings of entire proposi- tions. In every network layer, the capsules in- teract with each other and with representations of words in the sentence. Each iteration re- sults in updated proposition embeddings and updated predictions about the SRL structure. Our model substantially outperforms the non- refinement baseline model on all 7 CoNLL- 2019 languages and achieves state-of-the-art results on 5 languages (including English) for dependency SRL. We analyze the types of mistakes corrected by the refinement proce- dure. For example, each role is typically (but not always) filled with at most one argument. Whereas enforcing this approximate constraint is not useful with the modern SRL system, it- erative procedure corrects the mistakes by cap- turing this intuition in a flexible and context- sensitive way.1 1 Introduction The task of semantic role labeling (SRL) in- volves the prediction of predicate-argument struc- ture, i.e., both the identification of arguments and labeling them with underlying semantic roles. The shallow semantic structures have been shown beneficial in many natural language processing (NLP) applications, including information extrac- 1Code: https://github.com/DalstonChen/CapNetSRL. signing the contract injurypreventedHitifrom An x Initial Refined Gold signer (A0) signer (A0) document (A1) sign.01 sign.01 signer (A0) document (A1) Figure 1: An example predicate-argument structure: green and red arcs denote correct and wrong predic- tions, respectively. tion (Christensen et al., 2011), question answer- ing (Eckert and Neves, 2018) and machine trans- lation (Marcheggiani et al., 2018). In this work, we focus on the dependency ver- sion of the SRL task (Surdeanu et al., 2008). An example of a dependency semantic-role structure is shown in Figure 1. Edges in the graph are marked with semantic roles, whereas predicates (typically verbs or nominalized verbs) are anno- tated with their senses. Intuitively, there are many restrictions on poten- tial predicate-argument structures. Consider the ‘role uniqueness’ constraint: each role is typically, but not always, realized at most once. For ex- ample, predicates have at most one agent. Simi- larly, depending on a verb class, only certain sub- categorization patterns are licensed. Nevertheless, rather than modeling the interaction between ar- gument labeling decisions, state-of-the-art seman- tic role labeling models …

Content truncated to 500 words