Bridge-Language Capitalization Inference in Western Iranian: Sorani, Kurmanji, Zazaki, and Tajik

Bridge-Language Capitalization Inference in Western Iranian: Sorani, Kurmanji, Zazaki, and Tajik Patrick Littell, David Mortensen, Kartik Goyal, Chris Dyer, Lori Levin Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213, USA plittell@cs.cmu.edu, dmortens@cs.cmu.edu, kartikgo@cs.cmu.edu, cdyer@cs.cmu.edu, lsl@cs.cmu.edu Abstract In Sorani Kurdish, one of the most useful orthographic features in named-entity recognition – capitalization – is absent, as the language’s Perso-Arabic script does not make a distinction between uppercase and lowercase letters. We describe a system for deriving an inferred capitalization value from closely related languages by phonological similarity, and illustrate the system using several related Western Iranian languages. Keywords: Kurdish, named-entity recognition, phonology 1. Introduction In constructing a named-entity recognition system for So- rani Kurdish (Gautier, 1998; Thackston, 2006; Walther and Sagot, 2010; Esmaili and Salavati, 2013), a low-resource Western Iranian language written in a Perso-Arabic script, we were faced with a dilemma: one of the most use- ful orthographic features in named-entity recognition— capitalization—is absent in Perso-Arabic writing. However, within the Western Iranian family there are sev- eral languages, including Kurmanji Kurdish, Zazaki, and Tajik, that are written in Latin or Cyrillic scripts and there- fore do feature capitalization. This article details the pro- cess we developed and the challenges we faced in attempt- ing to infer a “surrogate” capitalization feature for Sorani named entity recognition based on Kurmaji, Zazaki, and Tajik sources. The question we are attempting to answer, and the process that we developed for answering it, is more general than just capitalization inference. Broadly, it is the question “How, for a language that is low-resourced with respect to feature F, can we infer values for F from material in a closely- related language?” This question, of bridge-language fea- ture inference, could equally be asked of various other fea- tures: the presence of vowels in scripts that only partially distinguish them, word class, tonal features, and, poten- tially, any lexical feature F. 2. Background 2.1. The larger project This research took place as part of a pilot project on linguis- tic rapid response for emergency situations. When given data in a low-resource language on which they have not worked, how much can a small, interdisciplinary team pro- cess in a very short (24- or 48-hour) timeframe? In par- ticular, what NLP milestones are possible within this time- frame when conventional textual and lexical resources are unavailable? This project is valuable not just because of its practical applications, but because it spurs investigation of poten- tial types of language resources that, in ordinary circum- stances, might be overlooked. We had little in the way of gold-standard annotated data, English-Sorani parallel text, or Sorani-language lexica and gazetteers1; as mentioned above, even the familiar feature of capitalization was ab- sent. This spurred us to consider what resources we might be able to adapt from “bridge languages”: closely related languages that are better-resourced.2 2.2. Languages 2.2.1. Kurdish Sorani (or “Central”) Kurdish is a language in the Iranian branch of the Indo-European family, spoken by 6.7 mil- lion people in Iraqi Kurdistan and …

Content truncated to 500 words