Simple Semantic Annotation and Situation Frames: Two Approaches to Basic Text Understanding in LORELEI

Simple Semantic Annotation and Situation Frames: Two Approaches to Basic Text Understanding in LORELEI Kira Griffitt, Jennifer Tracey, Ann Bies, Stephanie Strassel Linguistic Data Consortium. University of Pennsylvania 3600 Market Street, Suite 810, Philadelphia, PA. 19104 USA {kiragrif, garjen, bies, strassel}@ldc.upenn.edu Abstract We present two types of semantic annotation developed for the DARPA Low Resource Languages for Emerging Incidents (LORELEI) program: Simple Semantic Annotation (SSA) and Situation Frames (SF). Both of these annotation approaches are concerned with labeling basic semantic information relevant to humanitarian aid and disaster relief (HADR) scenarios, with SSA serving as a more general resource and SF more directly supporting the evaluation of LORELEI technology. Mapping between information in different annotation tasks is an area of ongoing research for both system developers and data providers. We discuss the similarities and differences between the two types of LORELEI semantic annotation, along with ways in which the general semantic information captured in SSA can be leveraged in order to recognize HADR-oriented information captured by SF. To date we have produced annotations for nineteen LORELEI languages; by the program’s end both SF and SSA will be available for over two dozen typologically diverse languages. Initially data is provided to LORELEI performers and to participants in NIST’s Low Resource Human Language Technologies (LoReHLT) evaluation series. After their use in LORELEI and LoReHLT evaluations the data sets will be published in the LDC catalog. Keywords: semantic annotation, information extraction, linguistic resources 1. Introduction Most of the world's languages are under-resourced for Human Language Technology (METANET, 2010; Rehm and Uszkoreit, 2012), but lack of resources does not correlate with lack of need for such technologies. The DARPA Low Resource Languages for Emerging Incidents (LORELEI) Program aims to advance the capabilities of NLP in low-resource languages, with a particular focus on using NLP to obtain situational awareness for an incident like a natural disaster involving a low-resource language within a short time of the emergence of that incident (DARPA, 2014). Linguistic Data Consortium (LDC) is building a variety of linguistic resources for nearly three dozen low resource languages for the LORELEI program (see Table 1). Representative Language Packs – consisting of large volumes of formal and informal monolingual and parallel (with English) text with a variety of manual annotations to support situational awareness, plus a lexicon, grammatical sketch and basic processing tools – are designed to enable research into language universals and cross-language projection. Incident Language Packs contain manually labeled evaluation data designed to test system performance on tasks related to situational awareness for one or more surprise languages per year that remain unknown until the start of the annual evaluation (Strassel and Tracey, 2016). Table 1: LORELEI Representative and Incident Languages This paper focuses on two semantic annotation tasks developed by LDC to support LORELEI research and evaluation: Simple Semantic Annotation (SSA) and Situation Frame (SF). Both SSA and SF label basic information relevant to humanitarian aid and disaster relief (HADR) scenarios. Situation Frame annotation directly corresponds to the LORELEI SF evaluation task, …

Content truncated to 500 words