An Approach Based on Multilingual Thesauri and Model Combination for Bilingual Lexicon Extraction

An approach based on multilingual thesauri and model combination for bilingual lexicon extraction Hervé Déjean, Éric Gaussier * Xerox Research Centre Europe 6, Chemin de Maupertuis, 38240 Meylan, France firstname.lastname@xrce.xerox.com Fatia Sadat Graduate School of Information Science Nara Institute of Science and Technology, Nara, Japan fatia-s@is.aist-nara.ac.jp Abstract This paper focuses on exploiting different models and methods in bilingual lexicon extraction, either from parallel or comparable corpora, in specialized domains. First, a special attention is given to the use of multilingual thesauri, and different search strategies based on such thesauri are investigated. Then, a method to combine the different models for bilingual lexicon extraction is presented. Our results show that the combination of the models significantly improves results, and that the use of the hierarchical information contained in our thesaurus, UMLS/MeSH, is of primary importance. Lastly, methods for bilingual terminology extraction and thesaurus enrichment are discussed. Introduction The growing availability of comparable corpora, through the Internet or via distribution agencies providing newspapers articles in different languages, has led researchers to develop methods to extract bilingual lexicons from such corpora, in order to enrich existing bilingual dictionaries, and help cross the language barrier for cross-language information retrieval. The results obtained thus far on comparable corpora, even though encouraging, are not completely satisfactory yet. (Fung, 2000) reports, for the Chinese-English language pair an accuracy of 76% to find the correct translation in the top 20 candidates, a figure we do not believe to be good enough to consider manual revision. Furthermore, the evaluation is carried out on 40 English words only. (Rapp, 1999) reaches 89% on the German-English language pair, when considering the top 10 candidates. If this figure is rather high, it was obtained on a set of 100 German words, which, even though not explicit in Rapp's paper, seem to be high frequency words, for which accurate and reliable statistics can be obtained. We want to show in this paper how previously proposed methods can be extended to and improved for specialized domains. In particular we will focus on the use and enrichment of multilingual thesauri, which, even though partially related they may be to the texts under consideration, are nonetheless an available and valuable resource for the task. We rely in this work on two main linguistic resources: a general bilingual dictionary (available through the ELRA consortium1) and a specialized multilingual thesaurus (the Medical Subject Headings, MeSH, provided through the metathesaurus Unified Medical Language System, UMLS2). Without anticipating too much on the linguistic preprocessing we use, it has to be noted that, unless otherwise stated, when we speak of a “word” we refer to a single (as opposed to compound), lexical word (as opposed to stop word). All our examples and experiments use the (German, English) language pair. 1 Context vectors: a basic building block Bilingual lexicon extraction from non-parallel but comparable corpora has been studied by a number of researchers, (Peters, 1995; Tanaka, 1996; Shahzad 1999; Rapp, 1999; Fung, 2000) among others. Their work relies on the assumption that …

Content truncated to 500 words