Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1506–1515 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling Diego Marcheggiani1 Ivan Titov1,2 1ILLC, University of Amsterdam 2ILCC, School of Informatics, University of Edinburgh marcheggiani@uva.nl ititov@inf.ed.ac.uk Abstract Semantic role labeling (SRL) is the task of identifying the predicate-argument struc- ture of a sentence. It is typically re- garded as an important step in the stan- dard NLP pipeline. As the semantic rep- resentations are closely related to syntac- tic ones, we exploit syntactic information in our model. We propose a version of graph convolutional networks (GCNs), a recent class of neural networks operating on graphs, suited to model syntactic de- pendency graphs. GCNs over syntactic de- pendency trees are used as sentence en- coders, producing latent feature represen- tations of words in a sentence. We ob- serve that GCN layers are complementary to LSTM ones: when we stack both GCN and LSTM layers, we obtain a substantial improvement over an already state-of-the- art LSTM SRL model, resulting in the best reported scores on the standard benchmark (CoNLL-2009) both for Chinese and En- glish. 1 Introduction Semantic role labeling (SRL) (Gildea and Juraf- sky, 2002) can be informally described as the task of discovering who did what to whom. For ex- ample, consider an SRL dependency graph shown above the sentence in Figure 1. Formally, the task includes (1) detection of predicates (e.g., makes); (2) labeling the predicates with a sense from a sense inventory (e.g., make.01); (3) identifying and assigning arguments to semantic roles (e.g., Sequa is A0, i.e., an agent / ‘doer’ for the corre- sponding predicate, and engines is A1, i.e., a pa- tient / ‘an affected entity’). SRL is often regarded Sequa makes and repairs jet engines. SBJ COORD OBJ CONJ NMOD ROOT A1 A1 A1 A0 A0 make.01 repair.01 engine.01 Figure 1: An example sentence annotated with se- mantic (top) and syntactic dependencies (bottom). as an important step in the standard NLP pipeline, providing information to downstream tasks such as information extraction and question answering. The semantic representations are closely re- lated to syntactic ones, even though the syntax- semantics interface is far from trivial (Levin, 1993). For example, one can observe that many arcs in the syntactic dependency graph (shown in black below the sentence in Figure 1) are mir- rored in the semantic dependency graph. Given these similarities and also because of availability of accurate syntactic parsers for many languages, it seems natural to exploit syntactic information when predicting semantics. Though historically most SRL approaches did rely on syntax (Thomp- son et al., 2003; Pradhan et al., 2005; Punyakanok et al., 2008; Johansson and Nugues, 2008), the last generation of SRL models put syntax aside in fa- vor of neural sequence models, namely LSTMs (Zhou and Xu, 2015; Marcheggiani et al., 2017), and outperformed syntactically-driven methods on standard benchmarks. We believe that one of the reasons for this radical …

Content truncated to 500 words