The IPR-cleared Corpus of Contemporary Written and Spoken Romanian Language

The IPR-cleared Corpus of Contemporary Written and Spoken Romanian Language Dan Tufiș, Verginica Barbu Mititelu, Elena Irimia, Ștefan Daniel Dumitrescu, Tiberiu Boroș Research Institute for Artificial Intelligence “Mihai Drăgănescu” 13, Calea 13 Septembrie, 050711, Bucharest, Romania {tufis, vergi, elena, sdumitrescu, tibi}@racai.ro Abstract The article describes the current status of a large national project, CoRoLa, aiming at building a reference corpus for the contemporary Romanian language. Unlike many other national corpora, CoRoLa contains only - IPR cleared texts and speech data, obtained from some of the country’s most representative publishing houses, broadcasting agencies, editorial offices, newspapers and popular bloggers. For the written component 500 million tokens are targeted and for the oral one 300 hours of recordings. The choice of texts is done according to their functional style, domain and subdomain, also with an eye to the international practice. A metadata file (following the CMDI model) is associated to each text file. Collected texts are cleaned and transformed in a format compatible with the tools for automatic processing (segmentation, tokenization, lemmatization, part-of-speech tagging). The paper also presents up-to-date statistics about the structure of the corpus almost two years before its official launching. The corpus will be freely available for searching. Users will be able to download the results of their searches and those original files when not against stipulations in the protocols we have with text providers. Keywords: corpus annotation, corpus design, IPR-clearing, metadata reference corpus, Romanian language 1. Introduction One of the most solicited resources out of our MetaNet4U distributions via Meta-Share platform (http://ws.racai.ro:9191/) is ROMBAC - the Romanian Balanced Corpus (Ion et al., 2012) containing 44,117,360 tokens covering four domains (News, Medical, Legal, Biographic and Fiction) and built around the earlier RoCo_News corpus (Tufiș and Irimia, 2006). The great feedback we received from the user community, as well as their growing need of larger and still balanced corpora encouraged us to start a more ambitious project, namely a reference corpus for contemporary Romanian, about 15 times larger, including a speech component and a treebank. The new project, called CoRoLa (Corpus of Contemporary Romanian Language), started in 2014 with a first version planned to be opened for the public at the end of 2017. The project, rated as a priority project of the Romanian Academy, has been joined by the Institute for Computer Science in Iaşi. Besides the two institutions which were commissioned by the Romanian Academy to run the CoRoLa project, the corpus developments is voluntarily contributed by linguist experts from the Linguistic Institute “Al. Philippide” of Iași and many Master and PhD students from University “A.I. Cuza” of Iași, University “Politehnica” of Bucharest and the University of Bucharest. Contemporary Romanian Language is the last phase in the evolution of the Romanian language, starting, according to the specialists, after the Second World War. Although initially planned to cover the period 1945-present, with two sub-periods (1945-1990, 1990-present), with clear differences mainly at the lexical level, that would have given this corpus a definite diachronic dimension, technical difficulties will restrict the …

Content truncated to 500 words