Memory-Based Morphological Analysis

Memory-Based Morphological Analysis Antal van den Bosch and Walter Daelemans ILK / Computational Linguistics Tilburg University {antalb,walter}@kub.nl} Abstract We present a general architecture for efficient and deterministic morphological analysis based on memory-based learning, and apply it to morphological analysis of Dutch. The system makes direct mappings from letters in context to rich categories that encode morphological boundaries, syntactic class labels, and spelling changes. Both precision and recall of labeled morphemes are over 84% on held-out dictionary test words and estimated to be over 93% in free text. 1 Introduction Morphological analysis is an essential compo- nent in language engineering applications rang- ing from spelling error correction to machine translation. Performing a full morphological analysis of a wordform is usually regarded as a segmentation of the word into morphemes, com- bined with an analysis of the interaction of these morphemes that determine the syntactic class of the wordform as a whole. The complexity of wordform morphology varies widely among the world's languages, but is regarded quite high even in the relatively simple cases, such as En- glish. Many wordforms in English and other western languages contain ambiguities in their morphological composition that can be quite in- tricate. General classes of linguistic knowledge that are usually assumed to play a role in this disambiguation process are knowledge of (i) the morphemes of a language, (ii) the morphotac- tics, i.e., constraints on how morphemes are al- lowed to attach, and (iii) spelling changes that can occur due to morpheme attachment. State-of-the art systems for morphological analysis of wordforms are usually based on two-level finite-state transducers (FSTS, Kosken- niemi (1983)). Even with the availability of sophisticated development tools, the cost and complexity of hand-crafting two-level rules is high, and the representation of concatenative compound morphology with continuation lexi- cons is difficult. As in parsing, there is a trade- off between coverage and spurious ambiguity in these systems: the more sophisticated the rules become, the more needless ambiguity they in- troduce. In this paper we present a learning approach which models morphological analysis (includ- ing compounding) of complex wordforms as se- quences of classification tasks. Our model, MBMA (Memory-Based Morphological Analy- sis), is a memory-based learning system (Stan- fill and Waltz, 1986; Daelemans et al., 1997). Memory-based learning is a class of induc- tive, supervised machine learning algorithms that learn by storing examples of a task in memory. Computational effort is invested on a "call-by-need" basis for solving new exam- ples (henceforth called instances) of the same task. When new instances are presented to a memory-based learner, it searches for the best- matching instances in memory, according to a task-dependent similarity metric. When it has found the best matches (the nearest neighbors), it transfers their solution (classification, label) to the new instance. Memory-based learn- ing has been shown to be quite adequate for various natural-language processing tasks such as stress assignment (Daelemans et al., 1994), grapheme-phoneme conversion (Daelemans and Van den Bosch, 1996; Van den Bosch, 1997), and part-of-speech tagging (Daelemans et al., 1996b). The …

Content truncated to 500 words