Phonotactic Modeling of Extremely Low Resource Languages

Phonotactic Modeling of Extremely Low Resource Languages Andrei Shcherbakov1 and Ekaterina Vylomova2 and Nick Thieberger1 1School of Languages and Linguistics 2Department of Computing and Information Systems The University of Melbourne ultrasparc@yandex.ru evylomova@gmail.com thien@unimelb.edu.au Abstract This paper presents a novel approach to low resource language modeling. Here we propose a model for word prediction which is based on multi-variant ngram ab- straction with weighted confidence level. We demonstrate a significant improvement in word recall over ”traditional” Kneser- Ney back-off model for most of the exam- ined low resource languages. 1 Introduction For a dictionary in the course of being created, a problem for the linguist/lexicographer is how to find words that are not already recorded in the dictionary. Linguists working on small languages may extract as many lexemes as they can from texts they record, and then add to that with elic- itation of items unlikely to appear in texts, es- pecially paradigmatic information (like verb in- flections, pronouns, kin terms and so on), place names, and biological names. They are now also using experimental stimuli to get at more nuanced meanings. However, once they have exhausted these sources, they need some way to discover other possible forms in the language. In this pa- per we discuss a method for creating possible word forms that can be confirmed by speakers (or not) as being words in the language. A general approach that has been used before is to apply the phonotac- tics to generate other possible forms (Prince and Tesar, 2004; Dell et al., 2000; Goldrick, 2004; Heinz, 2007). This was done with flip charts of possible sequences of phonemes, and then compu- tationally, generating forms on the basis of known permissible combinations.1 This task could also be viewed from the per- spective of machine learning, representing a par- ticular case of language modeling. We can state 1See e.g. http://billposer.org/Software/ WordGenerator.html the task as follows. The model gets an initially collected vocabulary of a given language as an in- put and has to predict likely undiscovered words as accurately as possible. There are many ap- proaches to this task. The most powerful ones known today are based on neural networks, includ- ing deep learning techniques (Sundermeyer et al., 2012; Kim et al., 2015; Sundermeyer et al., 2015; Hwang and Sung, 2016; Oparin et al., 2012). Un- fortunately, the ability of such algorithms to cap- ture knowledge depends heavily on the amount of training data, and generally they aren’t usable for a real language unless one provides at least several thousand training instances to them. Al- though, theoretically, such algorithms are able to learn many sophisticated rules not obvious even for a human analyst, the rules are hard to validate. Typically, for low resource languages we are not able to obtain a training vocabulary of sufficient size. In the current research, we consider a practi- cal task where an initial vocabulary of 300+ basic words has to be extended with more or less proper candidate words. For an optimal likelihood of hit- …

Content truncated to 500 words