Modeling Word Forms Using Latent Underlying Morphs and Phonology

Modeling Word Forms Using Latent Underlying Morphs and Phonology Ryan Cotterell and Nanyun Peng and Jason Eisner Department of Computer Science, Johns Hopkins University {ryan.cotterell,npeng1,eisner}@jhu.edu Abstract The observed pronunciations or spellings of words are often explained as arising from the “underlying forms” of their mor- phemes. These forms are latent strings that linguists try to reconstruct by hand. We propose to reconstruct them automatically at scale, enabling generalization to new words. Given some surface word types of a concatenative language along with the abstract morpheme sequences that they ex- press, we show how to recover consistent underlying forms for these morphemes, together with the (stochastic) phonology that maps each concatenation of underly- ing forms to a surface form. Our technique involves loopy belief propagation in a nat- ural directed graphical model whose vari- ables are unknown strings and whose con- ditional distributions are encoded as finite- state machines with trainable weights. We define training and evaluation paradigms for the task of surface word prediction, and report results on subsets of 7 languages. 1 Introduction How is plurality expressed in English? Compar- ing cats ([kæts]), dogs ([dOgz]), and quizzes ([kwIzIz]), the plural morpheme evidently has at least three pronunciations ([s], [z], [Iz]) and at least two spellings (-s and -es). Also, consider- ing singular quiz, perhaps the “short exam” mor- pheme has multiple spellings (quizz-, quiz-). Fortunately, languages are systematic. The re- alization of a morpheme may vary by context but is largely predictable from context, in a way that generalizes across morphemes. In fact, gener- ative linguists traditionally posit that each mor- pheme of a language has a single representation shared across all contexts (Jakobson, 1948; Ken- stowicz and Kisseberth, 1979, chapter 6). How- ever, this string is a latent variable that is never observed. Variation appears when the phonology of the language maps these underlying represen- tations (URs)—in context—to surface represen- tations (SRs) that may be easier to pronounce. The phonology is usually described by a grammar that may consist of either rewrite rules (Chomsky and Halle, 1968) or ranked constraints (Prince and Smolensky, 2004). We will review this framework in section 2. The upshot is that the observed words in a language are supposed to be explainable in terms of a smaller underlying lexicon of morphemes, plus a phonol- ogy. Our goal in this paper is to recover the lexicon and phonology (enabling generalization to new words). This is difficult even when we are told which morphemes are expressed by each word, be- cause the unknown underlying forms of the mor- phemes must cooperate properly with one another and with the unknown phonological rules to pro- duce the observed results. Because of these in- teractions, we must reconstruct everything jointly. We regard this as a problem of inference in a di- rected graphical model, as sketched in Figure 1. This is a natural problem for computational lin- guistics. Phonology students are trained to puzzle out solutions for small datasets by hand. Children apparently solve it at the scale …

Content truncated to 500 words