TextAnnotator: A UIMA Based Tool for the Simultaneous and Collaborative Annotation of Texts

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 891–900 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 891 TEXTANNOTATOR: A UIMA based tool for the simultaneous and collaborative annotation of texts Giuseppe Abrami, Manuel Stoeckel, Alexander Mehler Goethe University Frankfurt Robert-Mayer-Strasse 10 60325 Frankfurt am Main / Germany {abrami, mehler}@em.uni-frankfurt.de, manuel.stoeckel@stud.uni-frankfurt.de Abstract The annotation of texts and other material in the field of digital humanities and Natural Language Processing (NLP) is a common task of research projects. At the same time, the annotation of corpora is certainly the most time- and cost-intensive component in research projects and often requires a high level of expertise according to the research interest. However, for the annotation of texts, a wide range of tools is available, both for automatic and manual annotation. Since the automatic pre-processing methods are not error-free and there is an increasing demand for the generation of training data, also with regard to machine learning, suitable annotation tools are required. This paper defines criteria of flexibility and efficiency of complex annotations for the assessment of existing annotation tools. To extend this list of tools, the paper describes TEXTANNOTATOR, a browser-based, multi-annotation system, which has been developed to perform platform-independent multimodal annotations and annotate complex textual structures. The paper illustrates the current state of development of TEXTANNOTATOR and demonstrates its ability to evaluate annotation quality (inter-annotator agreement) at runtime. In addition, it will be shown how annotations of different users can be performed simultaneously and collaboratively on the same document from different platforms using UIMA as the basis for annotation. Keywords: UIMA, Annotation, Simultaneous and collaborative use, Inter-Annotator Agreement 1. Introduction The annotation of texts and other material in the field of digital humanities and Natural Language Processing (NLP) is a common task of research projects. At the same time, the annotation of corpora is certainly the most time- and cost-intensive component in research projects and often re- quires a high level of expertise according to the research interest. Such annotations enrich the data with additional information and subsequently allow them to be processed according to the needs of the project at hand. For this, over the past years, many annotation tools have been de- veloped for various tasks (for a recent process and feature oriented overview see (Finlayson and Erjavec, 2017); for an overview of multimodal tools see (Cassidy and Schmidt, 2017)). In addition to manual annotation tools, there is a proportionately larger number of automatic pre-processing tools. Despite the diversity of automated pre-processing applications (e.g. (Eckart de Castilho and Gurevych, 2014; Hemati et al., 2016)), a post-correction is often required because no complete coverage is achieved. For the efficient correction of errors in automatic pre-processing as well as for the generation of training data for machine learning and their efficient management, it is necessary to use a suitable annotation tool. This suitability of an annotation tool de- pends highly on the topic of the annotation task. In this context, it …

Content truncated to 500 words