Creating Large-Scale Multilingual Cognate Tables

Creating Large-Scale Multilingual Cognate Tables Winston Wu, David Yarowsky Department of Computer Science, Center for Language and Speech Processing Johns Hopkins University {wswu,yarowsky}@jhu.edu Abstract Low-resource languages often suffer from a lack of high-coverage lexical resources. In this paper, we propose a method to generate cognate tables by clustering words from existing lexical resources. We then employ character-based machine translation methods in solving the task of cognate chain completion by inducing missing word translations from lower-coverage dictionaries to fill gaps in the cognate chain, finding improvements over single language pair baselines when employing simple but novel multi-language system combination on the Romance and Turkic language families. For the Romance family, we show that system combination using the results of clustering outperforms weights derived from the historical-linguistic scholarship on language phylogenies. Our approach is applica- ble to any language family and has not been previously performed at such scale. The cognate tables are released to the research community. Keywords: cognates, clustering, transliteration eng lat fra ita spa por cat table mensa ? mensa mesa mesa ? table ? table tabella tabla tabela taula tableau tavolo tavola eng azj tat tuk tur uig uzn table stol östäl stol ? üstel stol table ? tablis tablisa tablo ? tablitsa Figure 1: Each row in the table is a cognate chain. The task of cognate chain completion is to fill in missing cells in the table. 1. Introduction Cognates are words in related languages that share a com- mon origin. For example, the Italian cavallo and French cheval both originated from the Latin caballus. Besides be- ing instrumental in historical linguistics, cognates find uses in many areas of NLP, including machine translation (Kon- drak et al., 2003; Nakov and Tiedemann, 2012) and lexicon induction (Mann and Yarowsky, 2001). We define the task of cognate chain completion, shown in Figure 1. Given multi-way aligned cognate table, a cog- nate “chain” is a group of cognates across a language family (represented as a single row). Chains may have empty cells due to dictionary gaps, denoted by a ?, and the task is to pre- dict these missing entries. Cognate chain completion is re- lated to the task of cognate transliteration, except that words in related languages (within the same row) can contribute to the hypothesis of a cell. For low-resource languages, gen- erating hypotheses for missing cognates has applications in alignment and resolving unknown words in machine trans- lation. In the field of linguistics, examining cognates across multiple related languages can shed light on how words are borrowed between languages. Cognate lists are not widely available for many languages and are time-consuming to create by hand. In many NLP-related applications, including the translating out-of- vocabulary words in machine translation, it is often not necessary that these words be true cognates in the linguis- tic sense, i.e. they are descendants of a common ances- tor (Ciobanu and Dinu, 2014). For example, names and loanwords are not technically considered cognates, though they behave as such. Rather, “cognates” only need …

Content truncated to 500 words