The Effect of Syntactic Representation on Semantic Role Labeling

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 393–400 Manchester, August 2008 The Effect of Syntactic Representation on Semantic Role Labeling Richard Johansson and Pierre Nugues Lund University, Sweden {richard, pierre}@cs.lth.se Abstract Almost all automatic semantic role label- ing (SRL) systems rely on a preliminary parsing step that derives a syntactic struc- ture from the sentence being analyzed. This makes the choice of syntactic repre- sentation an essential design decision. In this paper, we study the influence of syn- tactic representation on the performance of SRL systems. Specifically, we com- pare constituent-based and dependency- based representations for SRL of English in the FrameNet paradigm. Contrary to previous claims, our results demonstrate that the systems based on de- pendencies perform roughly as well as those based on constituents: For the ar- gument classification task, dependency- based systems perform slightly higher on average, while the opposite holds for the argument identification task. This is re- markable because dependency parsers are still in their infancy while constituent pars- ing is more mature. Furthermore, the re- sults show that dependency-based seman- tic role classifiers rely less on lexicalized features, which makes them more robust to domain changes and makes them learn more efficiently with respect to the amount of training data. 1 Introduction The role-semantic paradigm has a long and rich history in linguistics, and the NLP community has recently devoted much attention to develop- ing accurate and robust methods for performing c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. role-semantic analysis automatically (Gildea and Jurafsky, 2002; Litkowski, 2004; Carreras and Màrquez, 2005; Baker et al., 2007). It is widely conjectured that an increased SRL accuracy will lead to improvements in certain NLP applica- tions, especially template-filling systems. SRL has also been used in prototypes of more advanced semantics-based applications such as textual en- tailment recognition. It has previously been shown that SRL systems need a syntactic structure as input (Gildea and Palmer, 2002; Punyakanok et al., 2008). An im- portant consideration is then what information this input should represent. By habit, most systems for automatic role-semantic analysis have used Penn- style constituents (Marcus et al., 1993) produced by Collins’ (1997) or Charniak’s (2000) parsers. The influence of the syntactic formalism on SRL has only been considered in a few previous arti- cles. For instance, Gildea and Hockenmaier (2003) reported that a CCG-based parser gives improved results over the Collins parser. Dependency syntax has only received little at- tention for the SRL task, despite a surge of inter- est in dependency parsing during the last few years (Buchholz and Marsi, 2006). Early examples of dependency-based SRL systems, which used gold- standard dependency treebanks, include Žabokrt- ský et al. (2002) and Hacioglu (2004). Two stud- ies that compared the respective performances of constituent-based and dependency-based SRL sys- tems (Pradhan et al., 2005; Swanson and Gor- don, 2006), both using automatic parsers, reported that the constituent-based systems outperformed the dependency-based ones by a …

Content truncated to 500 words