SemEval-2015 Task 8: SpaceEval

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 884–894, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics SemEval-2015 Task 8: SpaceEval James Pustejovsky(1), Parisa Kordjamshidi(2,3), Marie-Francine Moens(2), Aaron Levine(1), Seth Dworman(1), Zachary Yocum(1) (1)Brandeis University, Waltham, MA (2)Katholieke Universiteit Leuven, Belgium (3)University of Illinois, Urbana/Champaign, IL {jamesp,zyocum,aclevine,sdworman}@brandeis.edu, Sien.Moens@cs.kuleuven.be, kordjam@illinois.edu Abstract Human languages exhibit a variety of strate- gies for communicating spatial information, including toponyms, spatial nominals, loca- tions that are described in relation to other lo- cations, and movements along paths. SpaceE- val is a combined information extraction and classification task with the goal of identify- ing and categorizing such spatial information. In this paper, we describe the SpaceEval task, annotation schema, and corpora, and evalu- ate the performance of several supervised and semi-supervised machine learning systems de- veloped with the goal of automating this task. 1 Introduction SpaceEval builds on the Spatial Role Labeling (SpRL) task introduced in SemEval 2012 (Kord- jamshidi et al., 2012) and used in SemEval 2013 (Kolomiyets et al., 2013). The base annotation scheme of the previous tasks was introduced in (Ko- rdjamshidi et al., 2010), with empirical practices in (Kordjamshidi et al., 2011; Kordjamshidi and Moens, 2015). While those previous tasks are similar in their goal, SpacEval adopts the annota- tion specification from ISOspace (Pustejovsky et al., 2011a; Moszkowicz and Pustejovsky, 2010; ISO/TC 37/SC 4/WG 2, 2014), a new standard for capturing spatial information. The SpRL in SemEval 2012 had a focus on the main roles of trajectors, landmarks, spatial indicators, and the links between these roles which form spatial relations. The formal semantics of the relations were considered at a course-grained level, consisting of three types: directional, regional (topological), and distal. The related annotated data, CLEF IAPR TC-12 Image Benchmark (Grubinger et al., 2006), contained mostly static spatial relations. In SemEval 2013, the SpRL task was extended to the recognition of motion indicators and paths, which are applied to the more dynamic spatial relations. Accordingly, the data set was expanded and the text from the Degree Confluence Project (Jarrett, 2013) webpages were annotated. SpaceEval extends the task in several dimensions, first by enriching the granularity of the semantics in both static and dynamic spatial configurations, and secondly by broadening the variety of annotated data and the domains considered. In SpaceEval the con- cept of place is distinguished from the concept of spatial entity as a fundamental typing distinction. That is, the roles of trajector (figure) and landmark (ground) are roles that are assigned to spatial enti- ties and places when occurring in spatial relations. Places, however, are inherently typed as such, and remain places, regardless of what spatial roles they may occupy. Obviously, an individual may assume multiple role assignments, and in both ISOspace and SpRL this is assumed to be the case. However, be- cause SpRL focuses on role assignment, it does not introduce the general concept of spatial entity. There are other differences in the relational schemas of SpRL and SpaceEval which can be eas- …

Content truncated to 500 words