Using a target language model for domain independent lexical disambiguation

MT Summit VII Sept. 1999 Using a Target Language Model for Domain Independent Lexical Disambiguation Jim Cowie, Yevgeny Ludovik, Sergei Nirenburg Computing Research Laboratory New Mexico State University USA (jcowie, eugene, sergei)@crl.nmsu.edu Abstract In this paper we describe a lexical disambigua- tion algorithm based on a statistical language model we call maximum likelihood disambigua- tion. The maximum likelihood method depends solely on the target language. The model was trained on a corpus of American English news- paper texts. Its performance was tested using output from a transfer based translation system between Turkish and English. The method is source language independent, and can be used for systems translating from any language into English. 1 Introduction One of the more persistent problems in machine translation is lexical disambiguation. We propose here a statistics-based solution to the problem in which disambiguation is based entirely on the target lan- guage. We created a statistical language model al- lowing us to compute the probability of any sequence of English words. We assume that any word in a source language may have several different transla- tions depending on its context. Given all the candidate translations of words in the source language, our sta- tistical model finds a word sequence in the target lan- guage that is the most probable translation of an input sentence. In this paper we first outline the problem, describe our approach, and introduce the statistical language model. Then we describe the algorithm for finding the most probable sentence from the set represented by the input ambiguous translation. Finally, we present some experimental results and discuss our future plans. 2 The Problem of Lexical Selection One source of translation ambiguity is crosslingual polysemy, that is, a situation when a word in the source language has multiple possible translations in the target language. This type of ambiguity is very common, as most source words may be translated into a set of "target language forms" in the target language. This is illustrated by the Spanish-to-English examples in Table 1. Headword Translations presidente President, prime minister, Presiding judge, presiding magistrate, mayor cima top, peak, summit, height Table 1: Example Translation Equivalents The transfer component of a typical MT system out- puts target language syntactic structures and features plus the translation of open class lexical material. Be- cause of ambiguity (lexical and other), the transfer module overgenerates and outputs many candidate translations of a source sentence. Among these candi- dates, some may provide a reader with enough infor- mation to understand the topic, but generally at best one or two would be considered to be appropriate translations. Current commercial MT systems have difficulty eliminating overgeneration, that is, selecting from a set of possible translations. Phrasal glossaries may help to produce correct translations for colloca- tions, but for individual words the typical method is to select as the translation the most frequent of the target language words in the candidate list. Although this approach is better than a random selection it obviously leads to problems, and results in …

Content truncated to 500 words