Semantic Relations between Text Segments for Semantic Storytelling: Annotation Tool - Dataset - Evaluation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4923–4932 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4923 Semantic Relations between Text Segments for Semantic Storytelling: Annotation Tool – Dataset – Evaluation Michael Raring, Malte Ostendorff, Georg Rehm DFKI GmbH, Alt-Moabit 91c, 10559 Berlin, Germany michael.raring@gmail.com, {malte.ostendorff, georg.rehm}@dfki.de Abstract Semantic Storytelling describes the goal to automatically and semi-automatically generate stories based on extracted, processed, classified and annotated information from large content resources. Essential is the automated processing of text segments extracted from different content resources by identifying the relevance of a text segment to a topic and its semantic relation to other text segments. In this paper we present an approach to create an automatic classifier for semantic relations between extracted text segments from different news articles. We devise custom annotation guidelines based on various discourse structure theories and annotate a dataset of 2,501 sentence pairs extracted from 2,638 Wikinews articles. For the annotation, we developed a dedicated annotation tool. Based on the constructed dataset, we perform initial experiments with Transformer language models that are trained for the automatic classification of semantic relations. Our results with promising high accuracy scores suggest the validity and applicability of our approach for future Semantic Storytelling solutions. Keywords: Semantic storytelling, Text classification, Discourse parsing, Wikinews 1. Introduction The ever-increasing size of information available dig- itally, and especially the growth of unstructured in- formation online, leads to the development of tech- nologies to curate, process and understand digital data (Moreno Schneider et al., 2017). Due to its semi- structured nature, it is a challenge for the machine to process the information automatically. Content cre- ators, scientists, and journalists write and publish ar- ticles and literary works based on researched informa- tion from a variety of (often digital) sources. They need domain-specific knowledge to verify content and facts, discuss different perspectives, and combine different sources. For journalists in particular, the ever-growing incoming stream of heterogeneous information, such as news articles, social media and press statements, is a major challenge. To better cope with this, knowledge workers rely on curation technologies to help them process, analyze, skim, sort, summarize, evaluate and present large amounts of digital content (Bois et al., 2017; Caselli and Vossen, 2017; van Meersbergen et al., 2017; Rehm et al., 2019; Rehm et al., 2021; Rehm et al., 2018; Linscheid et al., 2021). The long-term Semantic Storytelling vision describes the automatic and semi-automatic generation of stories based on extracted, processed, classified and annotated information from large content resources (Rehm et al., 2019; Rehm et al., 2020b; Rehm et al., 2021). Story- telling can be understood as a technique to order a se- ries of events in the world or to recognize meaningful patterns in natural language (Bruner, 1991). Hereby, Semantic Storytelling is one element of our long-term efforts to develop curation technologies as part of the QURATOR project (Rehm et al., 2020a). Our Seman- tic Storytelling approach is characterized by a stronger focus on …

Content truncated to 500 words