Discovering Morphological Paradigms from Plain Text Using a Dirichlet Process Mixture Model

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 616–627, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Discovering Morphological Paradigms from Plain Text Using a Dirichlet Process Mixture Model Markus Dreyer∗ SDL Language Weaver Los Angeles, CA 90045, USA mdreyer@sdl.com Jason Eisner Computer Science Dept., Johns Hopkins University Baltimore, MD 21218, USA jason@cs.jhu.edu Abstract We present an inference algorithm that orga- nizes observed words (tokens) into structured inflectional paradigms (types). It also natu- rally predicts the spelling of unobserved forms that are missing from these paradigms, and dis- covers inflectional principles (grammar) that generalize to wholly unobserved words. Our Bayesian generative model of the data ex- plicitly represents tokens, types, inflections, paradigms, and locally conditioned string edits. It assumes that inflected word tokens are gen- erated from an infinite mixture of inflectional paradigms (string tuples). Each paradigm is sampled all at once from a graphical model, whose potential functions are weighted finite- state transducers with language-specific param- eters to be learned. These assumptions natu- rally lead to an elegant empirical Bayes infer- ence procedure that exploits Monte Carlo EM, belief propagation, and dynamic programming. Given 50–100 seed paradigms, adding a 10- million-word corpus reduces prediction error for morphological inflections by up to 10%. 1 Introduction 1.1 Motivation Statistical NLP can be difficult for morphologically rich languages. Morphological transformations on words increase the size of the observed vocabulary, which unfortunately masks important generalizations. In Polish, for example, each lexical verb has literally 100 inflected forms (Janecki, 2000). That is, a single lexeme may be realized in a corpus as many different word types, which are differently inflected for person, number, gender, tense, mood, etc. ∗ This research was done at Johns Hopkins University as part of the first author’s dissertation work. It was supported by the Human Language Technology Center of Excellence and by the National Science Foundation under Grant No. 0347822. All this makes lexical features even sparser than they would be otherwise. In machine translation or text generation, it is difficult to learn separately how to translate, or when to generate, each of these many word types. In text analysis, it is difficult to learn lexical features (as cues to predict topic, syntax, semantics, or the next word), because one must learn a separate feature for each word form, rather than generalizing across inflections. Our engineering goal is to address these problems by mostly-unsupervised learning of morphology. Our linguistic goal is to build a generative probabilistic model that directly captures the basic representations and relationships assumed by morphologists. This model suffices to define a posterior distribution over analyses of any given collection of type and/or token data. Thus we obtain scientific data interpretation as probabilistic inference (Jaynes, 2003). Our computa- tional goal is to estimate this posterior distribution. 1.2 What is Estimated Our inference algorithm jointly reconstructs token, type, and grammar information about a language’s morphology. This has not previously been attempted. Tokens: We will tag each word token in a …

Content truncated to 500 words