Semantic Role Labeling Using Different Syntactic Views

Proceedings of the 43rd Annual Meeting of the ACL, pages 581–588, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Semantic Role Labeling Using Different Syntactic Views∗ Sameer Pradhan, Wayne Ward, Kadri Hacioglu, James H. Martin Center for Spoken Language Research, University of Colorado, Boulder, CO 80303 {spradhan,whw,hacioglu,martin}@cslr.colorado.edu Daniel Jurafsky Department of Linguistics, Stanford University, Stanford, CA 94305 jurafsky@stanford.edu Abstract Semantic role labeling is the process of annotating the predicate-argument struc- ture in text with semantic labels. In this paper we present a state-of-the-art base- line semantic role labeling system based on Support Vector Machine classifiers. We show improvements on this system by: i) adding new features including fea- tures extracted from dependency parses, ii) performing feature selection and cali- bration and iii) combining parses obtained from semantic parsers trained using dif- ferent syntactic views. Error analysis of the baseline system showed that approx- imately half of the argument identifica- tion errors resulted from parse errors in which there was no syntactic constituent that aligned with the correct argument. In order to address this problem, we com- bined semantic parses from a Minipar syn- tactic parse and from a chunked syntac- tic representation with our original base- line system which was based on Charniak parses. All of the reported techniques re- sulted in performance improvements. 1 Introduction Semantic Role Labeling is the process of annotat- ing the predicate-argument structure in text with se- ∗This research was partially supported by the ARDA AQUAINT program via contract OCG4423B and by the NSF via grants IS-9978025 and ITR/HCI 0086132 mantic labels (Gildea and Jurafsky, 2000; Gildea and Jurafsky, 2002; Gildea and Palmer, 2002; Sur- deanu et al., 2003; Hacioglu and Ward, 2003; Chen and Rambow, 2003; Gildea and Hockenmaier, 2003; Pradhan et al., 2004; Hacioglu, 2004). The architec- ture underlying all of these systems introduces two distinct sub-problems: the identification of syntactic constituents that are semantic roles for a given pred- icate, and the labeling of the those constituents with the correct semantic role. A detailed error analysis of our baseline system indicates that the identification problem poses a sig- nificant bottleneck to improving overall system per- formance. The baseline system’s accuracy on the task of labeling nodes known to represent semantic arguments is 90%. On the other hand, the system’s performance on the identification task is quite a bit lower, achieving only 80% recall with 86% preci- sion. There are two sources of these identification errors: i) failures by the system to identify all and only those constituents that correspond to semantic roles, when those constituents are present in the syn- tactic analysis, and ii) failures by the syntactic ana- lyzer to provide the constituents that align with cor- rect arguments. The work we present here is tailored to address these two sources of error in the identifi- cation problem. The remainder of this paper is organized as fol- lows. We first describe a baseline system based on the best published techniques. We then report on two sets of experiments using techniques …

Content truncated to 500 words