Rapid Development of Morphological Analyzers for Typologically Diverse Languages

Rapid Development of Morphological Analyzers for Typologically Diverse Languages Seth Kulick and Ann Bies Linguistic Data Consortium University of Pennsylvania 3600 Market Street, Suite 810 Philadelphia, PA 19104 {skulick,bies}@ldc.upenn.edu Abstract The Low Resource Language research conducted under DARPA’s Broad Operational Language Translation (BOLT) program required the rapid creation of text corpora of typologically diverse languages (Turkish, Hausa, and Uzbek) which were annotated with morphological information, along with other types of annotation. Since the output of morphological analyzers is a significant aid to morphological annotation, we developed a morphological analyzer for each language in order to support the annotation task, and also as a deliverable by itself. Our framework for analyzer creation results in tables similar to those used in the successful SAMA analyzer for Arabic (Maamouri et al., 2010), but with a more abstract linguistic level, from which the tables are derived. A lexicon was developed from available resources for integration with the analyzer, and given the speed of development and uncertain coverage of the lexicon, we assumed that the analyzer would necessarily be lacking in some coverage for the project annotation. Our analyzer framework was therefore focused on rapid implementation of the key structures of the language, together with accepting “wildcard” solutions as possible analyses for a word with an unknown stem, building upon our similar experiences with morphological annotation with Modern Standard Arabic and Egyptian Arabic. Keywords: Morphology, Lexicon, Morphological analyzer 1. Introduction The Low Resource Language research conducted un- der DARPA’s Broad Operational Language Translation (BOLT) program required the rapid creation of text cor- pora of typologically diverse languages (Turkish, Hausa, and Uzbek) which were annotated with morphological in- formation, along with other types of annotation. Since the output of morphological analyzers is a significant aid to morphological annotation, we developed a morphological analyzer for each language in order to support the annota- tion task, and also as a deliverable by itself. We developed a framework for creating an analyzer which results in tables similar to those used in the successful SAMA analyzer for Arabic (Maamouri et al., 2010), but with a more abstract linguistic level, from which the tables are derived. A lexi- con was developed from available resources for integration with the analyzer, and given the speed of development and uncertain coverage of the lexicon, we assumed that the an- alyzer would necessarily be lacking in some coverage for project annotation. Our analyzer framework was therefore focused on rapid implementation of the key structures of the language, together with accepting “wildcard” solutions as possible analyses for a word with an unknown stem, building upon our experience with morphological annota- tion with Modern Standard Arabic and Egyptian Arabic. In Section 2. we give an overview of the three interacting aspects of the analyzer infrastructure. In Section 3. we dis- cuss the framework for specifying the possible analyses for a language, and how it interacts with the separate lexicon development. In Section 4. we discuss further the compar- ison with SAMA, and in Section 5. some additional …

Content truncated to 500 words