An Empirical Evaluation of Annotation Practices in Corpora from Language Documentation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2778–2787 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2778 An empirical evaluation of annotation practices in corpora from language documentation Kilu von Prince, Sebastian Nordhoff Leibniz-Zentrum Allgemeine Sprachwissenschaft (ZAS Berlin) Schützenstr. 18, 10117 Berlin {vonprince, nordhoff}@leibniz-zas.de Abstract For most of the world’s languages, no primary data are available, even as many languages are disappearing. Throughout the last two decades, however, language documentation projects have produced substantial amounts of primary data from a wide variety of endangered languages. These resources are still in the early days of their exploration. One of the factors that makes them hard to use is a relative lack of standardized annotation conventions. In this paper, we will describe common practices in existing corpora in order to facilitate their future processing. After a brief introduction of the main formats used for annotation files, we will focus on commonly used tiers in the widespread ELAN and Toolbox formats. Minimally, corpora from language documentation contain a transcription tier and an aligned translation tier, which means they constitute parallel corpora. Additional common annotations include named references, morpheme separation, morpheme-by-morpheme glosses, part-of-speech tags and notes. Keywords: endangered languages, annotations, parallel corpora 1. Background 1.1. Introduction Our knowledge about the world’s languages is heavily bi- ased towards large, official, literate languages (Dahl, 2015) from western, educated, industrialized, rich, democratic so- cieties (Henrich et al., 2010). For most of the languages spoken today, no substantial records of primary data are available (Simons and Fennig, 2017). Throughout the last two decades, language documentation has emerged as a new subfield of linguistics that is concerned with provid- ing repositories of primary data from less resourced and endangered languages. The corpora produced within this field represent some of the richest records of spoken lan- guage available today. As audio and video recordings with time-aligned transcriptions, translations into one or more widespread languages, detailed metadata, and often morpheme-by-morpheme glosses and part-of-speech tags, these resources hold an immense potential both for lin- guistic research and for the development of advanced nat- ural language processing (Good, 2011; Cox, 2011; Seifart, 2012). However, the exploration of these resources is still far from trivial. Even when deposited in one of the major archives, they can be hard to locate and hard to access due to techni- cal and legal barriers. Once located, users will have to make sense of the annotations, which do not adhere to a fixed set of conventions. There are, of course, good reasons for this lack of conventions. The corpus creators have been faced for the first time with an extreme variety of sociolinguis- tic settings, with structurally very different languages, and with speaker communities that each have their own needs and restrictions. At the same time, most language docu- menters have been working with a limited set of tools and workflows, and had similar goals in mind, so that certain annotation practices are sufficiently widespread to serve …

Content truncated to 500 words