Resources in Underrepresented Languages: Building a Representative Romanian Corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3291–3296 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3291 A Representative Corpus of the Romanian Language: Resources in Underrepresented Languages Ludmila Midrigan - Ciochina, Victoria Boyd, Lucila Sanchez Ortega, Diana Malancea- Malac, Doina Midrigan, David P. Corina University of California, Davis, Diana.malancea@gmail.com, doinamidrigan@gmail.com {lmidrigan, vaboyd, lsan, dpcorina} @ ucdavis.edu Abstract The effort in the field of Linguistics to develop theories that aim to explain language-dependent effects on language processing is greatly facilitated by the availability of reliable resources representing different languages. This project presents a detailed description of the process of creating a large and representative corpus in Romanian – a relatively under-resourced language with unique structural and typological characteristics, that can be used as a reliable language resource for linguistic studies. The decisions that have guided the construction of the corpus, including the type of corpus, its size and component resource files are discussed. Issues related to data collection, data organization and storage, as well as characteristics of the data included in the corpus are described. Currently, the corpus has approximately 5,500,000 tokens originating from written text and 100,000 tokens of spoken language. it includes language samples that represent a wide variety of registers (i.e. written language - 16 registers and 5 registers of spoken language), as well as different authors and speakers Keywords: Romanian, corpus representativeness, corpus design Introduction As a field, theoretical linguistics has moved from idealized descriptive accounts of language toward more explanatory theories that strive to ascertain the effects of language experience on language processing and human cognition in general. Along with this shift has come a greater appreciation of the need to consider linguistic phenomena in all the world’s languages. This effort has been greatly aided by the development of computational approaches and digital resources in many of the world’s languages. The availability of reliable, easy to access resources is crucial for different linguistics subfields. Bender (2009, 2016) argues for the importance of “linguistic knowledge” referring to ways in which different languages vary in their structure, for the development of language-independent natural language processing systems. Corpora are an essential tool in studying typological patterns across and within languages, documenting and preserving natural and endangered languages and developing computational tools for language processing (Gippert, Himmelmann and Mosel, 2006; Crystal 2000; Bradley and Bradley 2002; etc.). Hence, the project described in this paper aims at creating a such resource in one of the world’s languages that remains relatively underrepresented and understudied to date. This paper provides a detailed description of the process of creating a representative corpus in Romanian, an Eastern – Romance language with unique grammatical and typological characteristics. The corpus is not only a large but also a balanced repository of written and spoken samples of language in Romanian, that can be used as a reliable tool in linguistic studies. Historically, Romanian has lacked large collections of empirical linguistic data, which has made it difficult …

Content truncated to 500 words