Generating a Yiddish Speech Corpus, Forced Aligner and Basic ASR System for the AHEYM Project

Generating a Yiddish Speech Corpus, Forced Aligner and Basic ASR System for the AHEYM Project Małgorzata E. ´Cavar, Damir Cavar, Dov-Ber Kerler, Anya Quilitsch Indiana University Bloomington, IN, USA {mcavar,dcavar,dkerler,aquilitz}@indiana.edu Abstract To create automatic transcription and annotation tools for the AHEYM corpus of recorded interviews with Yiddish speakers in Eastern Europe we develop initial Yiddish language resources that are used for adaptations of speech and language technologies. Our project aims at the development of resources and technologies that can make the entire AHEYM corpus and other Yiddish resources more accessible to not only the community of Yiddish speakers or linguists with language expertise, but also historians and experts from other disciplines or the general public. In this paper we describe the rationale behind our approach, the procedures and methods, and challenges that are not specific to the AHEYM corpus, but apply to all documentary language data that is collected in the field. To the best of our knowledge, this is the first attempt to create a speech corpus and speech technologies for Yiddish. This is also the first attempt to work out speech and language technologies to transcribe and translate a large collection of Yiddish spoken language resources. Keywords: Yiddish, Speech and Language Corpus, Speech Technology 1. Introduction Yiddish, in spite of the long tradition of scholarship, can be counted nowadays among languages with an uncertain future. This is – among others – due to its dispersed sta- tus and low number of speakers. Ethnologue1 reported no more than 1.5 mil. speakers of Eastern Yiddish worldwide in 1991 (with some 600,000 speakers in 1991 and 200,000 speakers in the USA estimated by the Modern Language Association, the latter number undated) and 50,000 speak- ers world-wide for Western Yiddish in year 2000. From the time when this data was collected, the numbers of speakers in Eastern Europe diminished dramatically. What is disap- pearing is the cultural heritage but also linguistic diversity of the multitude of Yiddish dialects. Another reason for classifying Yiddish as a potentially endangered, and surely under-resourced, is that it belongs to the group of languages with only limited speech and language technologies re- sources. Crucially, Yiddish recordings stored in archives usually lack word-by-word or sentence-by-sentence tran- scription, which limits the potential audience that can use and work with the recordings to researchers who can un- derstand Yiddish and – importantly – its dialectal variants. Further, any large scale search and analysis of such col- lected data is impractical without transcription. In this re- spect, Yiddish is like hundreds of other endangered lan- guages where researchers in the past decades frantically collected audio recordings of the language but only limited annotation for the recordings is usually available. In this paper we describe efforts to create an initial set of language resources and tools that later serve as the basis from which we bootstrap annotations and transcriptions for already existing hundreds of hours of Yiddish audio and video recordings. We focus in this project on the resources provided by the …

Content truncated to 500 words