Looking at Unbalanced Specialized Comparable Corpora for Bilingual Lexicon Extraction

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1284–1293, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Looking at Unbalanced Specialized Comparable Corpora for Bilingual Lexicon Extraction Emmanuel Morin and Amir Hazem Universit´e de Nantes, LINA UMR CNRS 6241 2 rue de la houssini`ere, BP 92208, 44322 Nantes Cedex 03, France {emmanuel.morin,amir.hazem}@univ-nantes.fr Abstract The main work in bilingual lexicon ex- traction from comparable corpora is based on the implicit hypothesis that corpora are balanced. However, the historical context- based projection method dedicated to this task is relatively insensitive to the sizes of each part of the comparable corpus. Within this context, we have carried out a study on the influence of unbalanced specialized comparable corpora on the quality of bilingual terminology extraction through different experiments. Moreover, we have introduced a regression model that boosts the observations of word co- occurrences used in the context-based pro- jection method. Our results show that the use of unbalanced specialized comparable corpora induces a significant gain in the quality of extracted lexicons. 1 Introduction The bilingual lexicon extraction task from bilin- gual corpora was initially addressed by using par- allel corpora (i.e. a corpus that contains source texts and their translation). However, despite good results in the compilation of bilingual lex- icons, parallel corpora are scarce resources, es- pecially for technical domains and for language pairs not involving English. For these reasons, research in bilingual lexicon extraction has fo- cused on another kind of bilingual corpora com- prised of texts sharing common features such as domain, genre, sampling period, etc. without hav- ing a source text/target text relationship (McEnery and Xiao, 2007). These corpora, well known now as comparable corpora, have also initially been introduced as non-parallel corpora (Fung, 1995; Rapp, 1995), and non-aligned corpora (Tanaka and Iwasaki, 1996). According to Fung and Che- ung (2004), who range bilingual corpora from par- allel corpora to quasi-comparable corpora going through comparable corpora, there is a continuum from parallel to comparable corpora (i.e. a kind of filiation). The bilingual lexicon extraction task from com- parable corpora inherits this filiation. For instance, the historical context-based projection method (Fung, 1995; Rapp, 1995), known as the standard approach, dedicated to this task seems implicitly to lead to work with balanced comparable corpora in the same way as for parallel corpora (i.e. each part of the corpus is composed of the same amount of data). In this paper we want to show that the assump- tion that comparable corpora should be balanced for bilingual lexicon extraction task is unfounded. Moreover, this assumption is prejudicial for spe- cialized comparable corpora, especially when in- volving the English language for which many doc- uments are available due the prevailing position of this language as a standard for international scientific publications. Within this context, our main contribution consists in a re-reading of the standard approach putting emphasis on the un- founded assumption of the balance of the spe- cialized comparable corpora. In specialized do- mains, the …

Content truncated to 500 words