Two-step flow in bilingual lexicon extraction from unrelated corpora

Two-Step Flow in Bilingual Lexicon Extraction from Unrelated Corpora Rogelio Nazar1, Leo Wanner 2 and Jorge Vivaldi1 1 Institut Universitari de Lingüística Aplicada, Pompeu Fabra University Pl. de la Mercè 10-12. Barcelona. 2 Fundació Barcelona Media, Pompeu Fabra University Ocata 1. Barcelona. {rogelio.nazar; leo.wanner; jorge.vivaldi}upf.edu Abstract. This paper presents a language independent methodology for automatically extracting bilingual lexicon entries from the web without the need of resources like parallel or comparable corpora, POS tagging, nor an initial bilingual lexicon. It is suitable for specialized domains where bilingual lexicon entries are scarce. The input for the process is a corpus in the source language to use as example of real usage of the units we need to translate. It is a two-step flow process because first we extract single-word units from the source language and then the multi-word units where the initial single units are instantiated. For each of the multi-word units, we see if they appear in texts from the web in the target language. The unit of the target language that appears more frequently across the sets of multi-word units is usually the correct translation of the initial single-word source language entry. Keywords: Bilingual Lexicon Extraction, Specialized Terminology, Machine Translation, Corpus Linguistics, Knowledge-poor methods, statistical methods. 1 Introduction Strategies that involve the use of parallel corpora were among the first attempts to extract bilingual lexicons, using measures of statistical association to study the co- occurrence of pairs of entries in the aligned sentences ([1]; [2]; among others). This methodology has yielded accurate results. However, the shortcoming is that parallel corpora are not easy to compile, particularly in the case of specialized domains. There have also been a number of attempts to extract bilingual lexicons without the need of parallel corpora, but using bilingual dictionaries as seed words. In this line of research there are two main trends. The first one is represented by authors such as [3]; [4]; [5]; [6] and [7]. Briefly, most of these approaches involve a similarity metric between a word in the source language and a candidate for translation in the target language. The rationale behind this strategy is that both the source language word and its equivalent are supposed to share the same profile of co-occurrence, in the same 12th EAMT conference, 22-23 September 2008, Hamburg, Germany 140 manner that synonyms do ([8]; [9]; [10]). The process is then to study the units that co-occur significantly with the input word, and then try to translate as many as possible co-occurrents with the help of the initial bilingual lexicon. Once this information is gathered, the next step is to select as a candidate for translation the unit of the target language that co-occurs more often (in some corpus) with the greatest number of the translations obtained with the bilingual lexicon. The other trend in the literature ([11]; [12]) is more related to the present proposal. In order to obtain documents where equivalents co-occur, [11] exploited search engines using pairs of equivalent terms in Japanese and …

Content truncated to 500 words