Dependency-based Semantic Role Labeling of PropBank

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 69–78, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Dependency-based Semantic Role Labeling of PropBank Richard Johansson and Pierre Nugues Lund University, Sweden {richard, pierre}@cs.lth.se Abstract We present a PropBank semantic role label- ing system for English that is integrated with a dependency parser. To tackle the problem of joint syntactic–semantic analysis, the sys- tem relies on a syntactic and a semantic sub- component. The syntactic model is a projec- tive parser using pseudo-projective transfor- mations, and the semantic model uses global inference mechanisms on top of a pipeline of classifiers. The complete syntactic–semantic output is selected from a candidate pool gen- erated by the subsystems. We evaluate the system on the CoNLL- 2005 test sets using segment-based and dependency-based metrics. Using the segment-based CoNLL-2005 metric, our system achieves a near state-of-the-art F1 figure of 77.97 on the WSJ+Brown test set, or 78.84 if punctuation is treated consistently. Using a dependency-based metric, the F1 figure of our system is 84.29 on the test set from CoNLL-2008. Our system is the first dependency-based semantic role labeler for PropBank that rivals constituent-based systems in terms of performance. 1 Introduction Automatic semantic role labeling (SRL), the task of determining who does what to whom, is a use- ful intermediate step in NLP applications perform- ing semantic analysis. It has obvious applications for template-filling tasks such as information extrac- tion and question answering (Surdeanu et al., 2003; Moschitti et al., 2003). It has also been used in prototypes of NLP systems that carry out complex reasoning, such as entailment recognition systems (Haghighi et al., 2005; Hickl et al., 2006). In addi- tion, role-semantic features have recently been used to extend vector-space representations in automatic document categorization (Persson et al., 2008). The NLP community has recently devoted much attention to developing accurate and robust methods for performing role-semantic analysis automatically, and a number of multi-system evaluations have been carried out (Litkowski, 2004; Carreras and Màrquez, 2005; Baker et al., 2007; Surdeanu et al., 2008). Following the seminal work of Gildea and Juraf- sky (2002), there have been many extensions in ma- chine learning models, feature engineering (Xue and Palmer, 2004), and inference procedures (Toutanova et al., 2005; Surdeanu et al., 2007; Punyakanok et al., 2008). With very few exceptions (e.g. Collobert and Weston, 2007), published SRL methods have used some sort of syntactic structure as input (Gildea and Palmer, 2002; Punyakanok et al., 2008). Most sys- tems for automatic role-semantic analysis have used constituent syntax as in the Penn Treebank (Marcus et al., 1993), although there has also been much re- search on the use of shallow syntax (Carreras and Màrquez, 2004) in SRL. In comparison, dependency syntax has received relatively little attention for the SRL task, despite the fact that dependency structures offer a more transparent encoding of predicate–argument rela- tions. Furthermore, the few systems based on de- pendencies that have been presented have generally performed much worse than their constituent-based …

Content truncated to 500 words