BERTrade: Using Contextual Embeddings to Parse Old French
16272Authors
nan
Abstract
AbstractThe successes of contextual word embeddings learned by training large-scale language models, while remarkable, have mostly occurred for languages where significant amounts of raw texts are available and where annotated data in downstream tasks have a relatively regular spelling. Conversely, it is not yet completely clear if these models are also well suited for lesser-resourced and more irregular languages. We study the case of Old French, which is in the interesting position of having relatively limited amount of available raw text, but enough annotated resources to assess the relevance of contextual word embedding models for downstream NLP tasks. In particular, we use POS-tagging and dependency parsing to evaluate the quality of such models in a large array of configurations, including models trained from scratch from small amounts of raw text and models pre-trained on other languages but fine-tuned on Medieval French data.
Keywords
Full Content
Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 100–108 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 100 Introducing the CURLICAT Corpora: Seven-language Domain Specific Annotated Corpora from Curated Sources Tamás Váradi1, Bence Nyéki1, Svetla Koeva2, Marko Tadić3, Vanja Štefanec3, Maciej Ogrodniczuk4, Bartłomiej Nitoń4, Piotr Pęzik5, Verginica Barbu Mititelu6, Elena Irimia6, Maria Mitrofan6,Vasile Păiș6, Dan Tufiș6, Radovan Garabík7, Simon Krek8, Andraž Repar8 1Hungarian Research Centre for Linguistics, Budapest, Hungary, {varadi.tamas,nyeki.bence}@nytud.hu 2Institute for Bulgarian Language, Bulgarian Academy of Sciences, Sofia, Bulgaria, svetla@dcl.bas.bg 3University of Zagreb, Faculty of Humanities and Social Sciences, Zagreb, Croatia, {marko.tadic,vanja.stefanec}@ffzg.hr 4Institute of Computer Science, Polish Academy of Sciences, Warsaw, Poland, {maciej.ogrodniczuk,bartlomiej.niton}@ipipan.waw.pl 5University of Łódź, Poland, pezik@uni.lodz.pl 6RACAI, Bucharest, Romania, {vergi,elena,maria,vasile,tufis}@racai.ro 7Ľ. Štúr Institute of Linguistics, Slovak Academy of Sciences, Bratislava, Slovakia, garabik@kassiopeia.juls.savba.sk 8IJS, Ljubljana, Slovenia, simon.krek@ijs.si, andraz.repar@cjvt.si Abstract This article presents the current outcomes of the CURLICAT CEF Telecom project, which aims to collect and deeply annotate a set of large corpora from selected domains. The CURLICAT corpus includes 7 monolingual corpora (Bulgarian, Croatian, Hungarian, Polish, Romanian, Slovak and Slovenian) containing selected samples from respective national corpora. These corpora are automatically tokenized, lemmatized and morphologically analysed and the named entities annotated. The annotations are uniformly provided for each language specific corpus while the common metadata schema is harmonised across the languages. Additionally, the corpora are annotated for IATE terms in all languages. The file format is CoNLL-U Plus format, containing the ten columns specific to the CoNLL-U format and three extra columns specific to our corpora as defined by Varádi et al. (2020). The CURLICAT corpora represent a rich and valuable source not just for training NMT models, but also for further studies and developments in machine learning, cross-lingual terminological data extraction and classification. Keywords: national corpora, comparable corpora, domain corpora 1. Introduction The present paper introduces the CURLICAT corpora and related resources compiled in the CEF Telecom Action1 of the same name. The CEF Telecom project Curated Multilingual Language Resources for CEF.AT (CURLICAT)2 aims to enhance the eTranslation system3 developed by the European Commission through supplying seven large corpora consisting of subsets of national and/or reference corpora in Bulgaria, Croatia, Hungary, Poland, Romania, Slovakia and Slovenia. The structure of the paper is as follows. We describe the rationale and objectives of the work in section 2. The composition of the corpus in each of the seven languages is presented in section 3, whereas section 4 introduces the additional sources of data since it was expected that there will not be enough data available from the national corpora to meet the envisaged sizes of data for each domain. The format and annotation, as well as the metadata of the multilingual corpora are described in sections 5 and 6 respectively. In the section 7, future processing steps are described: IATE4 annotation, the issues of anonymisation and IPR protection. Some conclusions are given in section 8. 1 https://ec.europa.eu/inea/en/connecting-europe-facility/cef- telecom 2 http://curlicat.eu 3 https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/ eTranslation 4 https://iate.europa.eu/home 2. Rationale and …
Content truncated to 500 words
Labeled Text
AbstractThe successes of contextual word embeddings learned by training large-scale language models, while remarkable, have mostly occurred for languages where significant amounts of raw texts are available and where annotated data in downstream tasks have a relatively regular spelling. Conversely, it is not yet completely clear if these models are also well suited for lesser-resourced and more irregular languages. We study the case of Old French, which is in the interesting position of having relatively limited amount of available raw text, but enough annotated resources to assess the relevance of contextual word embedding models for downstream NLP tasks. In particular, we use POS-tagging and dependency parsing to evaluate the quality of such models in a large array of configurations, including models trained from scratch from small amounts of raw text and models pre-trained on other languages but fine-tuned on Medieval French data.
Publication Details
- Paper ID:
- 16272
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan