Building an MT Dictionary From Parallel Texts Based on Linguistic and Statistical Information

BUILDING AN MT I)ICTIONARY FROM PARAI~LEI~ TEXTS BASED ON LINGUISTIC AND STATISTICAL INIi'ORMATION Akira Kumano ltidcki ltirakawa R & D Center, Toshiba Corporation 1, Komukai Toshiba-cho, Saiwai-ku, Kawasaki, 210, JAPAN { km n,hirakawa} @ist.rdc.toshiba.co.jp Abstract A method for generating a machine translation (MT) dictionary from parallel texts is described. This method utilizes both statistical information and linguistic information to obtain corresponding words or phrases in parallel texts. By combining these two types of information, translation pairs which cannot be obtained by a linguistic-based method can be extntcted. Over 70% accurate transla- tions of compound nouns and over 50% of unknown words are obtained as tbe first candidate from small Japanese/Englisb parallel texts containing severe dis- tortions. 1 INTRODUCTION Parallel texts (corpora) are useful resources for acquiring a variety of linguistic knowledge (Dangan, 1991; Matsumoto, 1993), especially for machine translation systems which inherently require cus- tomizations. Translation dictionaries are, needless to say, the most basic and powerful knowledge source for improving and customizing translation systems. Our research interest lies in automatic gen- eration of translation dictionaries from parallel texts. In this perspective, finding corresponding words or phrases in bilingual texts will be the fun- damental factor for accurate translation. Statistics-based processing has proven to be very powerful for aligning sentences and words in paral- lel corpora (Brown, 1991; Gale, 1993; Chen, 1993). Kupiec proposes an Mgorithm for finding ~loun phras- es in bilingual corpora (Kupiec, 1993). In this algo o rithm, noui~-phrase candidates are extracted from tagged and aligned parallel texts using a noun phrase recognizer and tile correspondences of these nonn phrases are calculated based on the EM algorithm. Accuracy of around 90% has been attained for the Imndred highest ranking con'espondenccs. Statistics- based processing is effective when a relatively large amount of parallel texts is available, i.e. when high frequencies are obtained. On the other hand, existing linguistic knowl- edge can be used for finding corresponding words or phrases in parallel texts. For example, possible tar- get expressions for a source expression provided by a translation system (linguistic knowledge source) can be a key in searching the corresponding expressions in a corpus (Nogami, 1991; Katoh, 1993). Yanramo- to (1993) proposes a method for generating a transla- tion dictionary from Japanese/English parallel texts. In this method, English and Japanese com- pound noun phrases are extracted from parallel texts and their correspondences are searched by matching their possible translations generated by tile existing translation dictionary. However, acquirable noun phrases are limited by tile linguistic generative power of the translation dictionary. Fur- thernlore, tiffs method utilizes no sentence align- meat information which can reduce errors in finding noun phrase correspondences. This paper proposes a new method for generat- ing an MT dictionary from parallel texts. It uti- lizes both statistical and linguistic information to obtain corresponding words or phrases in parallel texts. By combining these two types of informa- tion, translation pairs which cannot be obtained by the above linguistic-based method can be extracted, and a highly accurate translation dictionary is gener- …

Content truncated to 500 words