A Robust Combination Strategy for Semantic Role Labeling

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 644–651, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics A Robust Combination Strategy for Semantic Role Labeling Llu´ıs M`arquez, Mihai Surdeanu, Pere Comas, and Jordi Turmo Technical University of Catalunya Barcelona, Spain {lluism,surdeanu,pcomas,turmo}@lsi.upc.edu Abstract This paper focuses on semantic role la- beling using automatically-generated syn- tactic information. A simple and robust strategy for system combination is pre- sented, which allows to partially recover from input parsing errors and to signif- icantly boost results of individual sys- tems. This combination scheme is also very flexible since the individual systems are not required to provide any informa- tion other than their solution. Extensive experimental evaluation in the CoNLL- 2005 shared task framework supports our previous claims. The proposed architec- ture outperforms the best results reported in that evaluation exercise. 1 Introduction The task of Semantic Role Labeling (SRL), i.e. the process of detecting basic event structures such as who did what to whom, when and where, has received considerable interest in the past few years (Gildea and Jurafsky, 2002; Surdeanu et al., 2003; Xue and Palmer, 2004; Pradhan et al., 2005a; Carreras and M`arquez, 2005). It was shown that the identification of such event frames has a signif- icant contribution for many Natural Language Pro- cessing (NLP) applications such as Information Ex- traction (Surdeanu et al., 2003) and Question An- swering (Narayanan and Harabagiu, 2004). Most current SRL approaches can be classified in one of two classes: approaches that take ad- vantage of complete syntactic analysis of text, pi- oneered by Gildea and Jurafsky (2002), and ap- proaches that use partial syntactic analysis, cham- pioned by previous evaluations performed within the Conference on Computational Natural Language Learning (CoNLL) (Carreras and M`arquez, 2004). The wisdom extracted from this volume of work in- dicates that full syntactic analysis has a significant contribution to the SRL performance, when using hand-corrected syntactic information. On the other hand, when only automatically- generated syntax is available, the quality of the in- formation provided through full syntax decreases because the state-of-the-art of full parsing is less robust and performs worse than the tools used for partial syntactic analysis. Under such real-world conditions, the difference between the two SRL ap- proaches (with full or partial syntax) is not that high. More interestingly, the two SRL strategies perform better for different semantic roles. For example, models that use full syntax recognize better agent and theme roles, whereas models based on partial syntax are better at recognizing explicit patient roles, which tend to be farther from the predicate and accu- mulate more parsing errors (M`arquez et al., 2005). The above observations motivate the work pre- sented in this paper. We introduce a novel semantic role labeling approach that combines several indi- vidual SRL systems. Intuitively, our approach can be separated in two stages: a candidate generation phase, where the solutions provided by several indi- vidual models are combined into a pool of candidate arguments, and …

Content truncated to 500 words