Efficient Data Selection for Bilingual Terminology Extraction from Comparable Corpora

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3401–3411, Osaka, Japan, December 11-17 2016. Efficient Data Selection for Bilingual Terminology Extraction from Comparable Corpora Amir Hazem1 Emmanuel Morin1 1 LINA - UMR CNRS 6241, Universit´e de Nantes, France {amir.hazem,emmanuel.morin}@univ-nantes.fr Abstract Comparable corpora are the main alternative to the use of parallel corpora to extract bilingual lexicons. Although it is easier to build comparable corpora, specialized comparable corpora are often of modest size in comparison with corpora issued from the general domain. Consequently, the observations of word co-occurrences which are the basis of context-based methods are un- reliable. We propose in this article to improve word co-occurrences of specialized comparable corpora and thus context representation by using general-domain data. This idea, which has been already used in machine translation task for more than a decade, is not straightforward for the task of bilingual lexicon extraction from specific-domain comparable corpora. We go against the mainstream of this task where many studies support the idea that adding out-of-domain docu- ments decreases the quality of lexicons. Our empirical evaluation shows the advantages of this approach which induces a significant gain in the accuracy of extracted lexicons. 1 Introduction Comparable corpora are the main alternative to the use of parallel corpora for the task of bilingual lexicon extraction, particularly in specialized and technical domains for which parallel texts are usually unavailable or difficult to obtain. Although it is easier to build comparable corpora (Talvensaari et al., 2007), specialized comparable corpora are often of modest size (around 1 million words) in comparison with general-domain comparable corpora (up to 100 million words) (Morin and Hazem, 2016). The main reason is related to the difficulty to obtain many specialized documents in a language other than English. For example, a single query on the Elsevier portal1 of documents containing in their title the term “breast cancer” returns 40,000 documents in English, where the same query returns 1,500 documents in French, 693 in Spanish and only 7 in German. The historical context-based approach dedicated to the task of bilingual lexicon extraction from com- parable corpora, and also known as the standard approach, relies on the simple observation that a word and its translation tend to appear in the same lexical contexts (Fung, 1995; Rapp, 1999). In this approach, each word is described by its lexical contexts in both source and target languages, and words in trans- lation relationship should have similar lexical contexts in both languages. To enhance bilingual lexicon induction, recent approaches use more sophisticated techniques such as topic models based on bilingual latent dirichlet allocation (BiLDA) (Vulic and Moens, 2013b; Vulic and Moens, 2013a) or bilingual word embeddings based on neural networks (Gouws et al., 2014; Chandar et al., 2014; Vulic and Moens, 2015; Vulic and Moens, 2016) (approaches respectively noted: Gouws, Chandar and BWESG+cos). All these approaches require at least sentence-aligned/document aligned parallel data (BiLDA, Gouws, Chandar) or non-parallel document-aligned data at the topic level (BWESG+cos). Since specialized comparable corpora are …

Content truncated to 500 words