Optimizing disambiguation in Swahili

Optimizing disambiguation in Swahili Arvi HURSKAINEN Institute for Asian and African Studies Box 59 FI-00014 University of Helsinki Arvi.Hurskainen@helsinki.fi Abstract It is argued in this paper that an optimal solution to disambiguation is a combination of linguistically motivated rules and resolution based on probability or heuristic rules. By disambiguation is here meant ambiguity resolution on all levels of language analysis, including morphology and semantics. The discussion is based on Swahili, for which a comprehensive analysis system has been developed by using two-level description in morphology and constraint grammar formalism in disambiguation. Particular attention is paid to optimising the use of different solutions for achieving maximal precision with minimal rule writing. 1 Introduction In ambiguity resolution of natural language, both explicit linguistic information and probability calculation have been used as basic approaches. In early experiments usually only one strategy was applied, so that ambiguity resolution was performed either with the help of linguistic rules, or through probability calculation. Advanced approaches make use of both strategies, and they differ mainly in what kind of role each of these two methods has in the system (Wilks and Stevenson 1998, Stevenson and Wilks 2001). Sources of structured data, such as the WordNet (Miller 1990; Resnik 1998b; Banerjee and Pedersen 2002), have also been made use of. It is commonly known that the more comprehensive the description of language is, the more ambiguous is the interpretation of individual words1. Ambiguity occurs between word classes, between variously inflected word- forms, and above all, between various meanings of a word. A fairly large number2 of words in different word categories have more than one clearly distinguished meaning. Semantic disambiguation tends to be the hardest part of the disambiguation process, largely because of the fact that in semantics there are few distinguishable categories that could be used for generalising disambiguation rules. Below I shall describe a method where the use of linguistic rules and probability has been optimised with minimal loss of linguistic precision. Morphological description is carried out in the framework of two-level formalism3. After having been under development for 19 years (Hurskainen 1992, 1996), the parser of Swahili has now reached a phase where the recall as well as the precision4 is close to 100% in unrestricted standard Swahili text. 1 By word is here meant any string of characters, excluding punctuation marks and diacritics. Also, multi-word concepts, if they are handled as single entities, are considered words. 2 I do not consider it meaningful to present statistical details of ambiguity, because, when semantic glosses are included, the borderline between real ambiguity and such ambiguity as is found between synonyms and near-synonyms is vague. 3 The development environment for designing the morphological parser was provided by Lingsoft and Kimmo Koskenniemi (1983). 4 The criterion of precision in morphological analysis is considered fulfilled if one of the readings of a word is correct in the context concerned, and all other readings are grammatically correct analyses in some other context. Disambiguation rules, as well as the rules for …

Content truncated to 500 words