Creating a Translation Matrix of the Bible’s Names Across 591 Languages

Creating a Translation Matrix of the Bible’s Names Across 591 Languages Winston Wu1*, Nidhi Vyas2*, David Yarowsky1 1Johns Hopkins University, 2Carnegie Mellon University {wswu,yarowsky}@jhu.edu, nkvyas@cs.cmu.edu Abstract For many of the world’s languages, the Bible is the only significant bilingual, or even monolingual, text, making it a unique training re- source for tasks such as translation, named entity analysis, and transliteration. Given the Bible’s small size, however, the output of standard word alignment tools can be extremely noisy, making downstream tasks difficult. In this work, we develop and release a novel resource of 1129 aligned Bible person and place names across 591 languages, which was constructed and improved using several approaches including weighted edit distance, machine-translation-based transliteration models, and affixal induction and transformation models. Our models out- perform a widely used word aligner on 97% of test words, showing the particular efficacy of our approach on the impactful task of broadly multilingual named-entity alignment and translation across a remarkably large number of world languages. We further illustrate the utility of our translation matrix for the multilingual learning of name-related affixes and their semantics as well as transliteration of named entities. Keywords: Bible, alignment, named entities, translation, transliteration 1. Introduction In a statistical machine translation (SMT) pipeline, word alignment is important for extracting phrase translations. However, for low-resource languages with very little data, these alignments may be extremely noisy or may not ex- ist at all. Thus, improving the quality of word alignments leads to more accurate phrase pairs, which in turn improves the quality of an SMT system (Och and Ney, 2003; Fraser and Marcu, 2006). For many low-resource languages, the Bible is the only text available, making it a valuable re- source to train machine translation (MT) systems. This paper focuses on the translation and transliteration of named entities from the Bible, which are a rich resource for study- ing lexical borrowing (Tsvetkov and Dyer, 2016), since they are usually borrowed between languages rather than trans- lated1(Whitney, 1881; Moravcsik, 1978; Myers-Scotton, 2002). Like cognates, names are often phonetically or orthograph- ically similar across languages, which make them suited for training transliteration systems, in contrast to words which may just be translations of each other. In addition, especially for low-resource languages, names can be an in- valuable source of information for learning morphemes and their semantics. However, finding their optimal translation is a challenging task, due to various reasons, including low oc- currence counts and because certain names have high trans- lation entropy or are translated into their localized proper names. In this work, we present: (1) our creation of a novel resource of 1129 English Bible named entities aligned across 591 languages; (2) novel methods to produce and clean the resource; (3) challenges and findings in this process; and (4) potential use cases of our resource. Our Bible names transla- tion matrix is available at github.com/wswu/trabina. We believe this resource will be of great linguistic importance in studying low resource languages and will be applicable in several areas such as …

Content truncated to 500 words