Bilingual Lexicon Generation Using Non-Aligned Signatures

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 98–107, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Bilingual Lexicon Generation Using Non-Aligned Signatures Daphna Shezaf Institute of Computer Science Hebrew University of Jerusalem daphna.shezaf@mail.huji.ac.il Ari Rappoport Institute of Computer Science Hebrew University of Jerusalem arir@cs.huji.ac.il Abstract Bilingual lexicons are fundamental re- sources. Modern automated lexicon gen- eration methods usually require parallel corpora, which are not available for most language pairs. Lexicons can be gener- ated using non-parallel corpora or a pivot language, but such lexicons are noisy. We present an algorithm for generating a high quality lexicon from a noisy one, which only requires an independent cor- pus for each language. Our algorithm in- troduces non-aligned signatures (NAS), a cross-lingual word context similarity score that avoids the over-constrained and inef- ficient nature of alignment-based methods. We use NAS to eliminate incorrect transla- tions from the generated lexicon. We eval- uate our method by improving the quality of noisy Spanish-Hebrew lexicons gener- ated from two pivot English lexicons. Our algorithm substantially outperforms other lexicon generation methods. 1 Introduction Bilingual lexicons are useful for both end users and computerized language processing tasks. They provide, for each source language word or phrase, a set of translations in the target language, and thus they are a basic component of dictio- naries, which also include syntactic information, sense division, usage examples, semantic fields, usage guidelines, etc. Traditionally, when bilingual lexicons are not compiled manually, they are extracted from par- allel corpora. However, for most language pairs parallel bilingual corpora either do not exist or are at best small and unrepresentative of the general language. Bilingual lexicons can be generated using non- parallel corpora or pivot language lexicons (see Section 2). However, such lexicons are noisy. In this paper we present a method for generating a high quality lexicon given such a noisy one. Our evaluation focuses on the pivot language case. Pivot language approaches deal with the scarcity of bilingual data for most language pairs by relying on the availability of bilingual data for each of the languages in question with a third, pivot, language. In practice, this third language is often English. A naive method for pivot-based lexicon genera- tion goes as follows. For each source headword1, take its translations to the pivot language using the source-to-pivot lexicon, then for each such transla- tion take its translations to the target language us- ing the pivot-to-target lexicon. This method yields highly noisy (‘divergent’) lexicons, because lexi- cons are generally intransitive. This intransitivity stems from polysemy in the pivot language that does not exist in the source language. For ex- ample, take French-English-Spanish. The English word spring is the translation of the French word printemps, but only in the season of year sense. Further translating spring into Spanish yields both the correct translation primavera and an incorrect one, resorte (the elastic object). To cope with the issue of divergence due to lex- ical intransitivity, we present an algorithm for …

Content truncated to 500 words