The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies

CoNLL 2008: Proceedings of the 12th Conference on Computational Natural Language Learning, pages 159–177 Manchester, August 2008 The CoNLL-2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies Mihai Surdeanu†,⋆ Richard Johansson‡ Adam Meyers⋄ Llu´ıs M`arquez†† Joakim Nivre‡‡,⋆⋆ †: Barcelona Media Innovation Center, mihai.surdeanu@barcelonamedia.org ⋆: Yahoo! Research Barcelona, mihais@yahoo-inc.com ‡: Lund University, richard@cs.lth.se ⋄: New York University, meyers@cs.nyu.edu ††: Technical University of Catalonia, lluism@lsi.upc.edu ‡‡: V¨axj¨o University, joakim.nivre@vxu.se ⋆⋆: Uppsala University, joakim.nivre@lingfil.uu.se Abstract The Conference on Computational Natu- ral Language Learning is accompanied ev- ery year by a shared task whose purpose is to promote natural language processing applications and evaluate them in a stan- dard setting. In 2008 the shared task was dedicated to the joint parsing of syntactic and semantic dependencies. This shared task not only unifies the shared tasks of the previous four years under a unique dependency-based formalism, but also ex- tends them significantly: this year’s syn- tactic dependencies include more informa- tion such as named-entity boundaries; the semantic dependencies model roles of both verbal and nominal predicates. In this pa- per, we define the shared task and describe how the data sets were created. Further- more, we report and analyze the results and describe the approaches of the participat- ing systems. 1 Introduction In 2004 and 2005 the shared tasks of the Confer- ence on Computational Natural Language Learn- ing (CoNLL) were dedicated to semantic role la- beling (SRL), in a monolingual setting (English). In 2006 and 2007 the shared tasks were devoted to the parsing of syntactic dependencies, using cor- pora from up to 13 languages. The CoNLL-2008 shared task1 proposes a unified dependency-based c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. 1http://www.yr-bcn.es/conll2008 formalism, which models both syntactic depen- dencies and semantic roles. Using this formalism, this shared task merges both the task of syntactic dependency parsing and the task of identifying se- mantic arguments and labeling them with semantic roles. Conceptually, the 2008 shared task can be divided into three subtasks: (i) parsing of syntactic dependencies, (ii) identification and disambigua- tion of semantic predicates, and (iii) identification of arguments and assignment of semantic roles for each predicate. Several objectives were addressed in this shared task: • SRL is performed and evaluated using a dependency-based representation for both syntactic and semantic dependencies. While SRL on top of a dependency treebank has been addressed before (Hacioglu, 2004), our approach has several novelties: (i) our constituent-to-dependency conversion strat- egy transforms all annotated semantic argu- ments in PropBank and NomBank not just a subset; (ii) we address propositions centered around both verbal (PropBank) and nominal (NomBank) predicates. • Based on the observation that a richer set of syntactic dependencies improves seman- tic processing (Johansson and Nugues, 2007), the syntactic dependencies modeled are more complex than the ones used in the previous CoNLL shared tasks. For example, we now include apposition links, dependencies de- rived from named entity (NE) structures, and better modeling of long-distance grammatical relations. • …

Content truncated to 500 words