Automatic Extraction of Morphological Lexicons from Morphologically Annotated Corpora

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1032–1043, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Automatic Extraction of Morphological Lexicons from Morphologically Annotated Corpora Ramy Eskander, Nizar Habash, Owen Rambow Center for Computational Learning Systems Columbia University {reskander,habash,rambow}@ccls.columbia.edu Abstract We present a method for automatically learn- ing inflectional classes and associated lem- mas from morphologically annotated corpora. The method consists of a core language- independent algorithm, which can be opti- mized for specific languages. The method is demonstrated on Egyptian Arabic and Ger- man, two morphologically rich languages. Our best method for Egyptian Arabic pro- vides an error reduction of 55.6% over a sim- ple baseline; our best method for German achieves a 66.7% error reduction. 1 Introduction Morphological lexicons specify all inflected forms for each lexeme; in a language with rich morphol- ogy, such a resource can be important for natural lan- guage processing (NLP) tasks in order to limit data sparseness. For example, a morphological lexicon is an important component of a morphological tagger or of a part-of-speech (POS) tagger for languages with rich morphology.1 Traditionally, a morpholog- ical lexicon has been created through painstaking lexicographic and morphological analysis of the lan- guage, drawing on unannotated corpora. Recently, new approaches have emerged. Fully or largely un- supervised approaches cannot link surface forms to morphosyntactic features and are thus not suited for building morphological lexicons. This problem is overcome by approaches that use explicit linguistic knowledge. In this paper, we investigate using exist- ing morphologically annotated corpora. In a mor- phologically annotated corpus, the words in natu- rally occurring texts or transcribed speech are anno- tated for the correct morphological analysis (includ- 1Note that a full-form morphological lexicon is functionally equivalent to a morphological analyzer-generator, since one can be used to create the other. ing, of course, core POS) in context. While there has been much work on computational morphology, to our knowledge this is the first paper to study the question of how to extract morphological lexicons from morphologically annotated corpora, and how to determine how much annotation is needed. In this paper, we assume a corpus with each word an- notated with morphosyntactic features and with a lemma which tells us what lexeme the word form is part of. The task is to predict the correspondence between a word form and its lemma and morpholog- ical features. This paper makes two contributions. First, we introduce an algorithm that learns unseen forms by analogy. This algorithm is language-independent. It incrementally merges complementary paradigm in- formation about different lexemes into more abstract and more informative inflectional classes. Second, we explore how to model stems, and we propose a generalization of the Semitic root-and-template modeling. We use Egyptian Arabic (EGY), and Ger- man (GER) as our test languages. We test on corpus data, in order to simulate a standard real-world ap- plication. The baseline just uses the word forms seen in training and does not predict any unseen forms. Our …

Content truncated to 500 words