Distributed Representations for Unsupervised Semantic Role Labeling

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2482–2491, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Distributed Representations for Unsupervised Semantic Role Labeling Kristian Woodsend and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB k.woodsend@ed.ac.uk, mlap@inf.ed.ac.uk Abstract We present a new approach for unsuper- vised semantic role labeling that lever- ages distributed representations. We in- duce embeddings to represent a predi- cate, its arguments and their complex in- terdependence. Argument embeddings are learned from surrounding contexts involv- ing the predicate and neighboring argu- ments, while predicate embeddings are learned from argument contexts. The in- duced representations are clustered into roles using a linear programming formu- lation of hierarchical clustering, where we can model task-specific knowledge. Experiments show improved performance over previous unsupervised semantic role labeling approaches and other distributed word representation models. 1 Introduction In recent years, an increasing body of work has been devoted to learning distributed word repre- sentations and their successful usage in numerous tasks and real-world applications. Examples in- clude language modeling (Collobert et al., 2011; Mikolov et al., 2013c; Mnih and Kavukcuoglu, 2013), paraphrase detection (Socher et al., 2011a), sentiment analysis (Socher et al., 2011b; Kalch- brenner et al., 2014), and most notably machine translation (Kalchbrenner and Blunsom, 2013; Cho et al., 2014; Auli et al., 2013). Distributed word representations (also known as word embed- dings) are trained by predicting the contexts in which the words or phrases occur. In this paper, we present a new approach for unsupervised semantic role labeling that leverages distributed representations. The goal of semantic role labeling is to discover the relations that hold between a predicate and its arguments in a given input sentence (e.g., “who” did “what” to “whom”, “when”, “where”, and “how”). 1. [The burglar]A0 [broke]V [the window]A1. 2. [The window]A1 [broke]V. In sentence (1), A0 represents the Agent of the breaking event, A1 represents the Patient (i.e., the physical object affected by the breaking event) and V determines the boundaries of the predicate. The semantic roles in the example are labeled in the style of PropBank (Palmer et al., 2005), a broad-coverage human-annotated corpus of se- mantic roles and their syntactic realizations. In the unsupervised case, the model must induce such la- bels from data without access to a predefined set of semantic roles. Role induction is commonly treated as a cluster- ing problem (Titov and Klementiev, 2012; Lang and Lapata, 2014). The input to the model are instances of arguments (e.g., window, the burglar in sentence (1)) and the output is a grouping of these instances into clusters such that each cluster contains arguments corresponding to a specific se- mantic role and each role corresponds to exactly one cluster. In other words, the syntactic repre- sentations of verbal predicates, and argument po- sitions are observable, whereas the associated se- mantic roles are latent and need to be inferred. The task is challenging due to its …

Content truncated to 500 words