Collecting Language Resources from Public Administrations in the Nordic and Baltic Countries

Collecting Language Resources from Public Administrations in the Nordic and Baltic Countries Andrejs Vasiļjevs, Rihards Kalniņš, Roberts Rozis, Aivars Bērziņš Tilde Vienibas gatve 75a, LV1004, Riga, Latvia {andrejs, rihards.kalnins, roberts.rozis, aivars.berzins}@tilde.com Abstract This paper presents Tilde’s work on collecting language resources from government institutions and other public administrations in the Nordic and Baltic countries. We introduce the activities and results of the European Language Resources Coordination (ELRC) action in this region, provide a synopsis of ELRC workshops held in all countries of the region, identify potential holders and donors of language data suitable for improving machine translation systems, and describe the language resources collected so far. We also describe several national projects and initiatives on sharing of language data accumulated in the public sector and creation of new language resources from this data. Opportunities and challenges in consolidating language data from the public sector are discussed, and related actions and regulatory initiatives are proposed. Keywords: language resources, national/international projects, parallel corpora, ELRC, machine translation, CEF eTranslation 1. Introduction Several large-scale projects and initiatives have been undertaken in this century to collect language resources (LR) and create LR repositories and infrastructures on a pan-European scale. A family of closely related projects co-funded by EC FP7 and ICT-PSP programmes (T4ME, CESAR, META- NORD, META-NET4U) succeeded by establishing the META-NET network of excellence, comprised of 60 research centres in 34 countries (Rehm&Uszkoreit, 2011), and creating META-SHARE1, an online repository of language resources distributed and automatically synchronized across 28 hosting nodes across Europe (Piperidis, 2012). The META-NORD project contributed to these developments by coordinating language resource collection and setting up META-SHARE repositories in all 8 Nordic and Baltic countries (Vasiļjevs et al., 2011). Nordic and Baltic countries are part of the CLARIN infrastructure2 (Wittenburg et al., 2010), which supports the sharing, use, and sustainability of language data and tools for research in the humanities and social sciences (Varadi et al., 2008). National consortiums are established and funded by the Nordic governments, such as CLARINO3 in Norway, FIN-CLARIN4 in Finland and CLARIN-DK5 in Denmark. The breath and usability of web data is demonstrated by the OPUS corpora6 (Tiedemann, 2012), in which parallel data is extracted and aligned from numerous web sources, covering from formal (e.g., legal texts) and highly technical (e.g., user interface strings, medicine descriptions) to informal (e.g., movie subtitles, conversational phrases) language. Consolidation activities have also targeted specific types of language resources. For instance, the EuroTermBank project supported by the EC eContent programme has consolidated heterogeneous multilingual terminology 1 http://www.meta-share.org/ 2 http://www.clarin.eu/ 3 http://clarin.w.uib.no/ 4 http://kitwiki.csc.fi/twiki/bin/view/FinCLARIN resources in a distributed online termbank (Vasiļjevs et al., 2008). Still, most of the LRs in these repositories are created by research institutions, or the translation departments of large international organisations (such as the European Commission or United Nations), and have resulted from research projects. At the same time, thousands of translation units are created every day by companies and public sector institutions. Acquisition of commercial data can be very costly (e.g., monetary of reciprocal data exchange …

Content truncated to 500 words