Unsupervised Consonant-Vowel Prediction over Hundreds of Languages

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1527–1536, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Unsupervised Consonant-Vowel Prediction over Hundreds of Languages Young-Bum Kim and Benjamin Snyder University of Wisconsin-Madison {ybkim,bsnyder}@cs.wisc.edu Abstract In this paper, we present a solution to one aspect of the decipherment task: the pre- diction of consonants and vowels for an unknown language and alphabet. Adopt- ing a classical Bayesian perspective, we performs posterior inference over hun- dreds of languages, leveraging knowledge of known languages and alphabets to un- cover general linguistic patterns of typo- logically coherent language clusters. We achieve average accuracy in the unsuper- vised consonant/vowel prediction task of 99% across 503 languages. We further show that our methodology can be used to predict more fine-grained phonetic dis- tinctions. On a three-way classification task between vowels, nasals, and non- nasal consonants, our model yields unsu- pervised accuracy of 89% across the same set of languages. 1 Introduction Over the past centuries, dozens of lost languages have been deciphered through the painstaking work of scholars, often after decades of slow progress and dead ends. However, several impor- tant writing systems and languages remain unde- ciphered to this day. In this paper, we present a successful solution to one aspect of the decipherment puzzle: auto- matically identifying basic phonetic properties of letters in an unknown alphabetic writing system. Our key idea is to use knowledge of the phonetic regularities encoded in known language vocabu- laries to automatically build a universal probabilis- tic model to successfully decode new languages. Our approach adopts a classical Bayesian per- spective. We assume that each language has an unobserved set of parameters explaining its observed vocabulary. We further assume that each language-specific set of parameters was itself drawn from an unobserved common prior, shared across a cluster of typologically related languages. In turn, each cluster derives its parameters from a universal prior common to all language groups. This approach allows us to mix together data from languages with various levels of observations and perform joint posterior inference over unobserved variables of interest. At the bottom layer (see Figure 1), our model assumes a language-specific data generat- ing HMM over words in the language vocabulary. Each word is modeled as an emitted sequence of characters, depending on a corresponding Markov sequence of phonetic tags. Since individual letters are highly constrained in their range of phonetic values, we make the assumption of one-tag-per- observation-type (e.g. a single letter is constrained to be always a consonant or always a vowel across all words in a language). Going one layer up, we posit that the language- specific HMM parameters are themselves drawn from informative, non-symmetric distributions representing a typologically coherent language grouping. By applying the model to a mix of lan- guages with observed and unobserved phonetic se- quences, the cluster-level distributions can be in- ferred and help guide prediction for unknown lan- guages and alphabets. We apply this approach to two small decipher- ment tasks: …

Content truncated to 500 words