Deriving Morphological Analyzers from Example Inflections

Deriving Morphological Analyzers from Example Inflections Markus Forsberg,1 Mans Hulden2 1Språkbanken, Department of Swedish, University of Gothenburg, Sweden 2Department of Linguistics, University of Colorado markus.forsberg@gu.se, mans.hulden@colorado.edu Abstract This paper presents a semi-automatic method to derive morphological analyzers from a limited number of example inflections suitable for languages with alphabetic writing systems. The system we present learns the inflectional behavior of morphological paradigms from examples and converts the learned paradigms into a finite-state transducer that is able to map inflected forms of previously unseen words into lemmas and corresponding morphosyntactic descriptions. We evaluate the system when provided with inflection tables for several languages collected from the Wiktionary. Keywords: learning morphology, paradigm induction, finite-state morphology 1. Introduction Morphological analysis tools that provide detailed mor- phosyntactic descriptions (MSDs) and lemmatization of ar- bitrary word forms in some language are widely held to be fundamental for good performance of many higher-level NLP applications (Tseng et al., 2005; Spoustová et al., 2007; Avramidis and Koehn, 2008; Zeman, 2008; Hulden and Francom, 2012), particularly for languages with rich inflec- tional and derivational morphology. Hand-built systems, often modeled as finite-state transducers, offer very reliable morphological parses, but are time-consuming to create, and require significant linguistic expertise from the devel- opers (Maxwell, 2015). In many cases, however, finding collated example inflections on a large scale for some lan- guage through resources such as the Wiktionary or simply by consulting a speaker of the language is far less laborious than the elaborate linguistic modeling required to produce a robust morphological analyzer. In this paper we address this by describing a tool for automatic generation of finite-state morphological analyzers from collections of example word forms together with their MSDs. These morphological analyzers are constructed in the classical finite-state paradigm (Beesley and Karttunen, 2003), are non-probabilistic, and are designed to be high- recall, and hence to return all linguistically plausible anal- yses and lemmas, much like a hand-built morphological analyzer would which is extended with a ‘guesser’ module. Our tool takes as input a set of words annotated with lemma and MSD, grouped into inflection tables, and pro- duces as output a morphological analyzer using the Xerox regular expression formalism (Karttunen et al., 1996), which we compile into a transducer with the open-source finite- state toolkit foma (Hulden, 2009).1 2. Background In this paper, we work with the idea that inflections and derivations of related word forms can be formally expressed as functions. Such a view has been commonly seen as an alternative to the finite-state morphology approach where 1Our code is freely available together with the training/test setup employed in this paper at github.com/marfors/ paradigmextract swim swam swum ring rang rung jump jumped jumped type typed typed x1 i x2 x1 a x2 x1 u x2 x1 i x2 x1 a x2 x1 u x2 x1 x1 ed x1 ed x1 x1 ed x1 ed x1 i x2 x1 a x2 x1 u x2 x1 x1 ed x1 ed x1 i x2 x1 a x2 x1 u x2 x1 x1 …

Content truncated to 500 words