Latvian National Corpora Collection – Korpuss.lv

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 5123–5129 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 5123 Latvian National Corpora Collection – Korpuss.lv Baiba Saul¯ıte∗, Roberts Dar ‘gis∗, Normunds Gr¯uz¯ıtis∗, Ilze Auzin¸a∗, Krist¯ıne Lev¯ane-Petrova∗, Lauma Pretkalnin¸a∗, Laura Rituma∗, P¯eteris Paikens∗, Art¯urs Znotin¸ˇs∗, Laine Strankale∗, Krist¯ıne Pokratniece∗, Ilm¯ars Poik¯ans∗, Guntis B¯arzdin¸ˇs∗, Inguna Skadin¸a∗, Anda Bakl¯ane†, Valdis Saulespur¯ens†, J¯anis Ziedin¸ˇs‡ ∗Institute of Mathematics and Computer Science, University of Latvia (IMCS UL) Raina bulv. 29, LV-1459, Riga, Latvia {baiba.valkovska, roberts.dargis, normunds.gruzitis}@lumii.lv †National Library of Latvia (NLL) Mukusalas iela 3, LV-1048, Riga, Latvia {anda.baklane, valdis.saulespurens}@lnb.lv ‡Culture Information Systems Centre (CISC) Terbatas iela 53-2, LV-1011, Riga, Latvia janis.ziedins@kis.gov.lv Abstract LNCC is a diverse collection of Latvian language corpora representing both written and spoken language and is useful for both linguistic research and language modelling. The collection is intended to cover diverse Latvian language use cases and all the important text types and genres (e.g. news, social media, blogs, books, scientific texts, debates, essays, etc.), taking into account both quality and size aspects. To reach this objective, LNCC is a continuous multi-institutional and multi-project effort, supported by the Digital Humanities and Language Technology communities in Latvia. LNCC includes a broad range of Latvian texts from the Latvian National Library, Culture Information Systems Centre, Latvian National News Agency, Latvian Parliament, Latvian web crawl, various Latvian publishers, and from the Latvian language corpora created by Institute of Mathematics and Computer Science and its partners, including spoken language corpora. All corpora of LNCC are re-annotated with a uniform morpho-syntactic annotation scheme which enables federated search and consistent linguistics analysis in all the LNCC corpora, as well as facilitates to select and mix various corpora for pre-training large Latvian language models like BERT and GPT. Keywords: Latvian language, text corpora, spoken language, Universal Dependencies, federated search 1. Introduction Latvian corpora are increasingly used for large lan- guage model pre-training such as LVBERT (Znotins and Barzdins, 2020), LitLatBERT (Ulcar et al., 2021) and GPT2-LV (Plenert, 2021). The crucial zero-shot learning capability of the large language models depends not only on the model and data size but also on the quality and the ency- clopaedic knowledge coverage of the training corpora, giv- ing rise to the term ‘GoodData’ (Press, 2021). The ground- breaking GPT-3 (Brown et al., 2020) language model was trained on 750GB of mostly English GoodData, while the GPT-SW3 model for the relatively “small” Swedish lan- guage – on a 100GB text collection (Ekgren et al., 2022). Latvian National Corpora Collection (LNCC) with its cur- rent aggregated size of nearly 10GB and broad coverage is a step towards Latvian GoodData suitable for training high quality Latvian large language models essential for various downstream tasks (particularly zero-shot NLU and NLG – the future of NLP). Since Latvian is a relatively less-resourced language and even smaller than Swedish, there will probably never be a single corpus available, in terms of size or quality, to pre- training equally large language models if …

Content truncated to 500 words