Composition of Word Representations Improves Semantic Role Labelling

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 407–413, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Composition of Word Representations Improves Semantic Role Labelling Michael Roth and Kristian Woodsend Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh {mroth,kwoodsen}@inf.ed.ac.uk Abstract State-of-the-art semantic role labelling systems require large annotated corpora to achieve full performance. Unfortunately, such corpora are expensive to produce and often do not generalize well across do- mains. Even in domain, errors are often made where syntactic information does not provide sufficient cues. In this pa- per, we mitigate both of these problems by employing distributional word repre- sentations gathered from unlabelled data. While straight-forward word representa- tions of predicates and arguments improve performance, we show that further gains are achieved by composing representa- tions that model the interaction between predicate and argument, and capture full argument spans. 1 Introduction The goal of semantic role labelling (SRL) is to discover the relations that hold between a pred- icate and its arguments in a given input sen- tence (e.g., “who” did “what” to “whom”, “when”, “where”, and “how”). This semantic knowl- edge at the predicate-argument level is required by inference-based NLP tasks in order to iden- tify meaning-preserving transformations, such as active/passive, verb alternations and nominaliza- tions. Several manually-build semantic resources, including FrameNet (Ruppenhofer et al., 2010) and PropBank (Palmer et al., 2005), have been developed with the goal of documenting and pro- viding examples of such transformations and how they preserve semantic role information. Given that labelled corpora are inevitably restricted in size and coverage, and that syntactic cues are not by themselves unambiguous or sufficient, the suc- cess of systems that automatically provide corre- sponding analyses has been limited in practice. Recent work on SRL has explored approaches that can leverage unlabelled data, following a semi-supervised (F¨urstenau and Lapata, 2012; Titov and Klementiev, 2012) or unsupervised learning paradigm (Abend et al., 2009; Titov and Klementiev, 2011). Unlabelled data provides ad- ditional statistical strength and can lead to more consistent models. For instance, latent representa- tions of words can be computed, based on distri- butional similarity or language modelling, which can be used as additional features during tradi- tional supervised learning. Although we would expect that extra features would improve classifier performance, this seems in part counter-intuitive. Just because one word has a specific representa- tion does not mean that it should be assigned a specific argument label. Instead, one would ex- pect a more complex interplay between predicate, argument and the context they appear in. In this paper, we investigate the impact of dis- tributional word representations for SRL. Initially, we augment the feature space with word repre- sentations for a predicate and its argument head. Furthermore, we use a compositional approach to model a representation of the full argument, by composing a joint representation of all words in the argument span, and we also investigate the in- teraction between predicate and argument, using a compositional …

Content truncated to 500 words