Don’t Annotate, but Validate: a Data-to-Text Method for Capturing Event Data

Don’t Annotate, but Validate: a Data-to-Text Method for Capturing Event Data Piek Vossen, Filip Ilievski, Marten Postma, Roxane Segers Vrije Universiteit Amsterdam De Boelelaan 1105, 1081 HV Amsterdam The Netherlands {piek.vossen, f.ilievski, m.c.postma, r.h.segers}@vu.nl Abstract In this paper, we present a new method to obtain large volumes of high-quality text corpora with event data for studying identity and reference relations. We report on the current methods to create event reference data by annotating texts and deriving the event data a posteriori. Our method starts from event registries in which event data is defined a priori. From this data, we extract so-called Microworlds of referential data with the Reference Texts that report on these events. This makes it possible to easily establish referential relations with high precision and at a large scale. In a pilot, we successfully obtained data from these resources with extreme ambiguity and variation, while maintaining the identity and reference relations and without having to annotate large quantities of texts word-by-word. The data from this pilot was annotated using an annotation tool created specifically in order to validate our method and to enrich the reference texts with event coreference annotations. This annotation process resulted in the Gun Violence Corpus, whose development process and outcome are described in this paper. Keywords: event coreference, text corpora, structured data 1. Introduction Events and entities are central to referential semantics. Se- mantic parsing of news articles not only concerns detect- ing the meaning of words and their relations, but especially establishing the referential relations to the outside world. For entities, it is straightforward what this referential world is. However, compared to entities, events are less tangi- ble (Guarino, 1999; Hovy et al., 2013) for various reasons: 1. we use a small vocabulary to name events, which results in large referential ambiguity 2. events are more open to in- terpretation and framing, which leads to more variation in making reference 3. events are less persistent in time than entities 4. each event has many idiosyncratic properties, e.g. unique participants playing different roles in a unique spa- tio-temporal context, making generalization harder. Due to these properties, textual data on events is more fragmented than textual data on entities. In news, events are mentioned during a very short period of time and they rapidly lose their news value (except for a few events like 9/11), whereas pop- ular entities tend to be mentioned across different texts over longer periods of time. We thus do not find a typical Zipfian distribution for events, with a few events that dominate the news (the head) and a long tail of low-frequent events, but a more even low-frequent distribution. Given this fragmented distribution, it is not surprising that NLP tasks on event detection, event relation detection, and event coreference are difficult, which is reflected by rel- atively low inter-annotator-agreements and small amounts of data with event annotations. Most corpora with event annotations do not consider how they relate to the same or similar events in the world, e.g. PropBank …

Content truncated to 500 words