Leveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora

Proceedings of the 27th International Conference on Computational Linguistics, pages 937–949 Santa Fe, New Mexico, USA, August 20-26, 2018. 937 Leveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora Amir Hazem Emmanuel Morin LS2N - UMR CNRS 6004, Universit´e de Nantes, France {amir.hazem,emmanuel.morin}@univ-nantes.fr Abstract Recent evaluations on bilingual lexicon extraction from specialized comparable corpora have shown contrasted performance while using word embedding models. This can be partially ex- plained by the lack of large specialized comparable corpora to build efficient representations. Within this context, we try to answer the following questions: First, (i) among the state-of-the-art embedding models, whether trained on specialized corpora or pre-trained on large general data sets, which one is the most appropriate model for bilingual terminology extraction? Second (ii) is it worth it to combine multiple embeddings trained on different data sets? For that purpose, we propose the first systematic evaluation of different word embedding models for bilingual termi- nology extraction from specialized comparable corpora. We emphasize how the character-based embedding model outperforms other models on the quality of the extracted bilingual lexicons. Further more, we propose a new efficient way to combine different embedding models learned from specialized and general-domain data sets. Our approach leads to higher performance than the best individual embedding model. 1 Introduction Bilingual lexicons are fundamental resources in multilingual natural language processing tasks such as machine translation (Och and Ney, 2003), cross-language information retrieval (Nie, 2010) or computer- assisted translation (Delpech, 2014). Because a manual compilation of bilingual lexicons requires sub- stantial human efforts, bilingual lexicons are automatically extracted from bilingual corpora. These corpora can be parallel or comparable data sets. Despite good results obtained when compiling bilingual lexicons from parallel corpora, the latter are scarce resources, especially for specialized and technical domains and for language pairs not involving English. In this context, comparable corpora are an inter- esting and practical alternative to the use of parallel corpora. Comparable corpora, which gather texts sharing common features such as domain, topic, discourse, etc. without having a parallel source text-target text relationship, allow access to the original vocabulary without falling under the influence of the human translation. Compiling a large comparable corpus is easier, especially for general language (Talvensaari et al., 2007). In contrast, specialized comparable corpora are traditionally of modest size due to the difficulty to obtain many specialized documents in a language other than English. Specialized comparable corpora have a size of around one million words whereas general-domain comparable corpora can gather several million words (Morin and Hazem, 2014). One way to overcome the small size of specialized comparable corpora is to associate external re- sources. These resources may be close specialized corpora (e.g. a breast cancer corpus may benefit from contexts derived from a more general oncology corpus), corpora of different types of discourse and gen- der (e.g. a corpus of popular science discourse supplementing a corpus of scientific discourse), corpora of general language or out-of-domain data. The main challenge is to know how to associate such resources with …

Content truncated to 500 words