Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3923–3933 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3923 Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages Silvia Severini, Ayyoob Imani, Philipp Dufter∗, Hinrich Schütze Center for Information and Language Processing LMU Munich, Germany silvia@cis.uni-muenchen.de Abstract Parallel corpora are ideal for extracting a multilingual named entity (MNE) resource, i.e., a dataset of names translated into multiple languages. Prior work on extracting MNE datasets from parallel corpora required resources such as large monolingual corpora or word aligners that are unavailable or perform poorly for underresourced languages. We present CLC-BN, a new method for creating an MNE resource, and apply it to the Parallel Bible Corpus, a corpus of more than 1000 languages. CLC-BN learns a neural transliteration model from parallel-corpus statistics, without requiring any other bilingual resources, word aligners, or seed data. Experimental results show that CLC-BN clearly outperforms prior work. We release an MNE resource for 1340 languages and demonstrate its effectiveness in two downstream tasks: knowledge graph augmentation and bilingual lexicon induction. Keywords: Low-resource,Multilinguality,Named Entities,Transliteration 1. Introduction Of the thousands of languages in the world, a very small portion is covered by language technologies (Joshi et al., 2020). Bird (2020) suggests a number of approaches to develop such technologies for low-resource languages. In this paper, our goal is to create a multilingual named entity (MNE) resource – by which we mean a dataset of names translated into multiple languages – for a large number of low-resource languages, in total more than a thousand. Named entities (NEs) are crucial for many language technologies and NLP applications, includ- ing text comprehension, question answering, informa- tion retrieval and relation extraction. In this paper, we demonstrate the effectiveness of our MNE resource in two downstream tasks: knowledge graph augmentation and bilingual lexicon induction. We extract our MNE resource from the Parallel Bible Corpus (PBC) (Mayer and Cysouw, 2014), a multiparal- lel corpus that covers more than 1300 languages. (Note however that we do not use Bible-specific features; therefore, our work is in principle applicable to any parallel corpus.) For some languages, PBC is the only available text (Wu et al., 2018). Multiparallel corpora contain sentence-level parallel text in more than two languages. Apart from PBC, JW300 (Agi´c and Vuli´c, 2019) and Tatoeba1 are two other examples of such cor- pora. While the amount of data per language provided by highly multiparallel corpora is usually small, this type of corpus plays an important part in compiling resources for low-resource languages. Creating a named entity resource is comparatively easy if sufficiently high-quality resources are available for ∗Now at Apple. 1https://tatoeba.org Kat: იერემია (ieremia) Heb:יִרְמְיָהוּ (yiremeyahu) Jpn: ェレムヤ (eremuya) Tam: எேரமியா (ereemiyaa) Ita: geremia Deu: jeremias jeremiah rachel Hin: राहेल (raahel) Fin: raakel Rus: рахи́ль (rakhil) Spa: raquel Kor: 라헬이 (raheli) Ell: ραχη%λ (rachel) Figure 1: Two NEs from our resource, each with a sample of translations in six different languages …

Content truncated to 500 words