Practical Bootstrapping of Morphological Analyzers

Practical Bootstrapping of Morphological Analyzers Kemal Oflazer 1,~ 1Department of Computer Engineering Bilkent University Bilkent, Ankara, 06533,Turkey ko@crl, nmsu. edu Sergei Nirenburg ~ 2Computing Research'Laboratory New Mexico State University Las Cruces, NM, 88003 sergei@crl, nmsu. edu Abstract This paper presents a semi-automatic technique for developing broad-coverage finite-state morphological analyzers for any language. It consists of three components-elicitation of linguistic information from humans, a machine learning bootstrapping scheme and a testing environment. The three components are ap- plied iteratively until a threshold of output quality is attained. The initial application of this technique is for morphology of low-density languages in the context of the Expedition project at NMSU CRL. This elicit- build-test technique compiles lexical and inflectional information elicited from a human into a finite state transducer lexicon and combines this with a sequence of morphographemic rewrite rules that is induced us- ing transformation-based learning from the elicited ex- amples. The resulting morphological analyzer is then tested against a test suite, and any corrections are fed back into the learning procedure that builds an im- proved analyzer. Introduction The Expedition project is devoted to fast "ramp-up" of machine translation systems from less studied, so-called "low-density" languages into English. One of the components that must be acquired and built during this process is a morphological analyzer for the source low-density language. Since we expect that the source language informant will not be well-versed in computational linguistics in general or in recent approaches to building morphological analyzers (e.g., [Koskenniemi, 1983], [Antworth. 1990], [Karttunen et al., 1992], [Karttunen, 1994]) and the operation of state-of-the-art finite state tools (e.g., [Karttunen. 1993], [Karttunen and Beesley, 1992], [Karttunen et al., 1996]) in particular, the generation of the morphological analyzer component has to be accomplished almost semi-automatically. The user must be guided through a knowledge elicitation proce- dure for the knowledge required for the morphological analyzer. This is accomplished using the elicitation component of Expedition, the Boas system. As this task is not easy, we expect that the development of the morphological analyzer will be an iterative process, whereby the human informant will revise and/or refine the information previously elicited based on the feedback from a test runs of the nascent analyzer. The work reported in this paper describes the use of machine learning in the process of building and re- fining morphological analyzers. The main use of ma- chine learning in our current approach is in the au- tomatic learning of formal rewrite or replace rules for morphographemic changes from the examples, provided by the informant. This subtask of accounting for such phenomena is perhaps one of the more complicated as- pects of building an analyzer and by automating it we expect to gain a certain improvement in productivity. There have been a number of studies on induc- ing morphographemic rules from a list of inflected words and a root word list. Johnson [1984] presents a scheme for inducing phonological rules from surface data, mainly in the context of studying certain aspects of language acquisition. The premise …

Content truncated to 500 words