SemEval-2013 Task 3: Spatial Role Labeling

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 255–262, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics SemEval-2013 Task 3: Spatial Role Labeling Oleksandr Kolomiyets†, Parisa Kordjamshidi†, Steven Bethard‡ and Marie-Francine Moens† †KU Leuven, Celestijnenlaan 200A, Heverlee 3001, Belgium ‡University of Colorado, Campus Box 594 Boulder, Colorado, USA Abstract Many NLP applications require information about locations of objects referenced in text, or relations between them in space. For ex- ample, the phrase a book on the desk contains information about the location of the object book, as trajector, with respect to another ob- ject desk, as landmark. Spatial Role Label- ing (SpRL) is an evaluation task in the infor- mation extraction domain which sets a goal to automatically process text and identify ob- jects of spatial scenes and relations between them. This paper describes the task in Se- mantic Evaluations 2013, annotation schema, corpora, participants, methods and results ob- tained by the participants. 1 Introduction Spatial Role Labeling at SemEval-2013 is the sec- ond iteration of the task, which was initially in- troduced at SemEval-2012 (Kordjamshidi et al., 2012a). The second iteration extends the previous work with an additional training corpus, which con- tains besides “static” spatial relations, annotated mo- tions. Motion detection is a novel task for annotating trajectors (objects, which are moving), landmarks (spatial context in which the motion is performed), motion indicators (lexical triggers which signals tra- jector’s motion), paths (a path along which the mo- tion is performed), directions (absolute or relative directions of trajector’s motion) and distances (a distance as a product of motion). For annotating motions the existing annotation scheme has been adapted with additional markables which are, all to- gether, described below. 2 Spatial Annotation Schema In this Section we describe the annotation format of spatial markables in text, and annotation guidelines for the annotators. 2.1 Spatial Annotation Format Building upon the previous work, we used the no- tions of trajectors, landmarks and spatial indicators as introduced by Kordjamshidi et al. (2010). In ad- dition, we further expanded the set of spatial roles labels with motion indicators, paths, directions and distances to capture fine-grained spatial semantics of static spatial relations (as the ones which do not in- volve motions), and to accommodate dynamic spa- tial relations (the ones which do involve motions). 2.1.1 Static Spatial Relations and their Roles Static spatial relations are defined as relations be- tween still objects, whereas one object plays a cen- tral role in the spatial scene, which is called tra- jector, and the second one plays a secondary role, and it is called landmark. In language, a spatial re- lation between two objects is usually implemented by a preposition (in, on, at, etc.) or a prepositional phrase (on top of, inside of, etc.). A static spatial relation is defined as a tuple that contains a trajector, a landmark and a spatial indica- tor. In the annotation schema, these annotations are defined as follows: …

Content truncated to 500 words