A Probabilistic Context-free Grammar for Disambiguation in Morphological Parsing

A Probabilistic Context-free Grammar for Disambiguation in Morphological Parsing Jos~e S. Heemskerk* Institute of Language Technology and Artificial Intelligence Tilburg University P.O. Box 90153, 5000 LE Tilburg The Netherlands E-mail: joseeh@kub.nl Abstract One of the major problems one is faced with when decomposing words into their constituent parts is ambiguity: the gen- eration of multiple analyses for one input word, many of which are implausible. In order to deal with ambiguity, the MOR- phological PArser MORPA is provided with a probabilistic context-free grammar (PCFG), i.e. it combines a "conventional" context-free morphological grammar to fil- ter out ungrammatical segmentations with a probability-based scoring function which determines the likelihood of each success- ful parse. Consequently, remaining analy- ses can be ordered along a scale of plausi- bility. Test performance data will show that a PCFG yields good results in morphologi- cal parsing. MORPA is a fully implemented parser developed for use in a text-to-speech conversion system. 1 Introduction MORPA is a MORphological PArser developed for use in the text-to-speech conversion system for Dutch, SPRAAKMAKER [van Leeuwen and te Lin- deft, 1993]. An important step in text-to-speech con- version is the generation of the correct phonemic re- presentation on the basis of the input text. As is well- known, phonemic transcriptions can not be derived *This work was carried out at the Phonetics Lab- oratory at Leiden University and supported by the Speech Technology Foundation, which is funded by the Netherlands Stimulation Project for Information Sciences, SPIN. directly from orthographic input in Dutch, as there is no one-to-one correspondence between graphemes and phonemes. Also, stress and the effects of most phonological rules are not reflected in orthography. A text-to-speech system therefore requires an intel- ligent method to convert the spelled words of the input sentence into a phonemic representation. As far as the pronunciation of words is concerned, it is impossible to list the entire vocabulary of the language, because language users have the ability to create new words and the vocabulary, as such, is in- definitely large. Daily newspapers, for instance, con- tain a large amount of newly formed words every day. Not all of these survive in the long run, but some of them do. Consider the examples in (1): (1) golfooriog 'gulf war' drugsbaron 'drugs baron' vredesmacht 'peacekeeping force' Because it is unfeasible to give the lexicon a daily update, this approach is not appropriate if the text- to-speech system is to convert unrestricted text. Assuming that newly created words will typically consist of already existing morphemes, and that new morphemes are added to the language only rarely, we can, however, use a lexicon in which all Dutch mor- phemes and their pronunciations are listed. Then complex words, such as the ones in (1), have to be decomposed into their constituent parts before their pronunciation can be looked up. Since the pronunciation of a word does not always consists of the concatenation of the pronunciation of the morphemes, because the pronunciation of mor- phemes can be modified in certain contexts, …

Content truncated to 500 words