A Multi-Orthography Parallel Corpus of Yiddish Nouns

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 948–952 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 948 A Multi­Orthography Parallel Corpus of Yiddish Nouns Jonne Sälevä Brandeis University Waltham, MA jonnesaleva@brandeis.edu Abstract Yiddish is a low­resource language belonging to the Germanic language family and written using the Hebrew alphabet. As a language, Yiddish can be considered resource­poor as it lacks both public accessible corpora and a widely­used standard orthography, with various countries and organizations influencing the spellings speakers use. While existing corpora of Yiddish text do exist, they are often only written in a single, potentially non­standard orthography, with no parallel version with standard orthography available. In this work, we introduce the first multi­orthography parallel corpus of Yiddish nouns built by scraping word entries from Wiktionary. We also demonstrate how the corpus can be used to bootstrap a transliteration model using the Sequitur­G2P grapheme­to­phoneme conversion toolkit to map between various orthographies. Our trained system achieves error rates between 16.79% and 28.47% on the test set, depending on the orthographies considered. In addition to quantitative analysis, we also conduct qualitative error analysis of the trained system, concluding that non­phonetically spelled Hebrew words are the largest cause of error. We conclude with remarks regarding future work and release the corpus and associated code under a permissive license for the larger community to use. Keywords: Yiddish, transliteration, diacritization, resource creation 1. Introduction Yiddish is an example of a resource­poor language, ex­ hibiting both general resource­scarcity and significant or­ thographic variation. In this paper, we present a multi­ orthography parallel Yiddish corpus based on Wiktionary. The corpus contains Yiddish nouns in several orthographic forms, and is intended to serve as a seed for further devel­ opment of transliteration and orthographic standardization systems. Our contributions are as follows: 1. We scrape all Yiddish nouns1, and generate a corpus containing parallel versions of each word in roman­ ized, YIVO, and diacriticless Chasidic orthography. 2. We bootstrap a transliteration model using Sequitur­ G2P (Bisani and Ney, 2008)2 and obtain results that are sufficiently performant to be used in downstream tasks. 3. Finally, we release the corpus and code under a permis­ sive license. Our hope is that future research will use them to create larger, more unified language resources for Yiddish. 2. Orthographic Challenges of Yiddish Yiddish has been traditionally been written using a modi­ fied version of the Hebrew alphabet (Jacobs, 2005). As a lower­resourced language, Yiddish presents several unique challenges to the development of language resources, and, ultimately, NLP systems. 2.1. Orthographic Variants In terms of modern usage, there are three main orthographic variants of Yiddish. The first major variant is the standard­ ized Hebrew spellings developed by the YIVO Institute for 1https://en.wiktionary.org/wiki/Category: Yiddish_nouns 2https://github.com/sequitur-g2p/sequitur-g2p Jewish Research3, as well as their gold standard roman­ ized forms. The other prominent variants are the Hebrew spellings used by the Orthodox/Chasidic Jewish commu­ nity, which frequently appear in community publications.4 For the most part, the YIVO orthographies …

Content truncated to 500 words