Curated Multilingual Language Resources for CEF AT (CURLICAT): overall view

Curated Multilingual Language Resources for CEF AT (CURLICAT): Overall View Tamás Váradi Research Institute for Linguistics, Budapest, Hungary varadi.tamas@nytud.hu Marko Tadić University of Zagreb, Faculty of Humanities and Social Sciences, Zagreb, Croatia marko.tadic@ffzg.hr Svetla Koeva Institute of Bulgarian Language, Bulgarian Academy of Sciences, Sofia, Bulgaria svetla@dcl.bas.bg Maciej Ogrodniczuk Institute of Computer Science, Polish Acad- emy of Sciences, Warsaw, Poland maciej.ogrodniczuk@ipipan.waw.pl Dan Tufiș RACAI, Romanian Academy, Bucharest, Romania tufis@racai.ro Radovan Garabík Ľ. Štúr Institute of Linguistics, Slovak Acad- emy of Sciences, Bratislava, Slovakia garabik@kassiopeia.juls.savba.sk Simon Krek, Andraž Repar Institute Jozef Stefan, Ljubljana, Slovenia simon.krek@ijs.si, repar.andraz@gmail.com Abstract The work in progress on the CEF action CURLICAT is presented. The general aim of the action is to compile curated monolingual datasets in seven languages of the consortium in domains of rele- vance to European Digital Service Infra- structures (DSIs) in order to enhance the eTranslation services. 1 Introduction The paper©presents the work in progress on the CEF action Curated Multilingual Language Resources for CEF AT (CURLICAT, which runs from 2020-06-01 till 2022-11-30). The aim of the action is to compile monolingual curated datasets in seven languages of the consortium (Bulgarian, Croatian, Hungarian, Polish, Romanian, Slovak, Slovenian) in domains of relevance to European Digital Service Infrastructures (DSIs) with a view to enhancing the eTranslation automated translation system. © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 2 Datasets The primary data come from national or refer- ence corpora of the above languages and it is planned to cover domains of interest for CEF DSIs such as eHealth, Europeana or eGovern- ment. When completed, the corpus will contain at least at least 2 million sentences from each language, i.e. 14 million sentences, estimated to number at least 140 million words, from domains including culture, health, science and econo- my/finances. For each language, it is expected to produce corpora in each of the above mentioned four domains with at least 500 000 sentences and 5 million words. In case that legally non-binding data with a clear licence allowing free redistribu- tion could not be found from the national corpora in the required quantities, additional data is in- cluded from other sources. 2.1 Annotation Apart from corpora being domain classified, data are linguistically annotated including sentence splitting, tokenisation, lemmatisation, part-of- speech/morphosyntactic-descriptor tagging, dep- endency parsing and NERC. The annotation follows the extended CoNLL-U Plus1 format presented by Váradi et al. (2020). Additionally, terms from the most recent version of the IATE terminological database are identified and annotated so that the language models built with the help of these corpora could take into account not only single words but also multi-word ex- pressions since these terms represent an addition- al layer of annotation in stand-off manner. With this additional annotation these corpora can serve as a valuable resource for terminological proc- essing as well. 2.2 Intellectual Propery Rights Issues and Anonymisation The data are technically and legally cleaned by either of two procedures: 1) inclusion of text …

Content truncated to 500 words