Combining String and Context Similarity for Bilingual Term Alignment from Comparable Corpora

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1701–1712, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Combining String and Context Similarity for Bilingual Term Alignment from Comparable Corpora Georgios Kontonatsios1,2 Ioannis Korkontzelos1,2 Jun’ichi Tsujii3 Sophia Ananiadou1,2 National Centre for Text Mining, University of Manchester, Manchester, UK1 School of Computer Science, University of Manchester, Manchester, UK2 Microsoft Research Asia, Beijing, China3 {gkontonatsios,ikorkontzelos,sananiadou}@cs.man.ac.uk jtsujii@microsoft.com Abstract Automatically compiling bilingual dictio- naries of technical terms from comparable corpora is a challenging problem, yet with many potential applications. In this paper, we exploit two independent observations about term translations: (a) terms are of- ten formed by corresponding sub-lexical units across languages and (b) a term and its translation tend to appear in similar lex- ical context. Based on the first observa- tion, we develop a new character n-gram compositional method, a logistic regres- sion classifier, for learning a string similar- ity measure of term translations. Accord- ing to the second observation, we use an existing context-based approach. For eval- uation, we investigate the performance of compositional and context-based methods on: (a) similar and unrelated languages, (b) corpora of different degree of compa- rability and (c) the translation of frequent and rare terms. Finally, we combine the two translation clues, namely string and contextual similarity, in a linear model and we show substantial improvements over the two translation signals. 1 Introduction Bilingual dictionaries of technical terms are re- sources useful for various tasks, such as computer- aided human translation (Dagan and Church, 1994; Fung and McKeown, 1997), Statistical Ma- chine Translation (Och and Ney, 2003) and Cross- Language Information Retrieval (Ballesteros and Croft, 1997). In the last two decades, researchers have focused on automatically compiling bilingual term dictionaries either from parallel (Smadja et al., 1996; Van der Eijk, 1993) or comparable cor- pora (Rapp, 1999; Fung and Yee, 1998). While parallel corpora contain the same sentences in two languages, comparable corpora consist of bilin- gual pieces of text that share some features, only, such as topic, domain, or time period. Comparable corpora can be constructed more easily than paral- lel corpora. Freely available, up-to-date, on-line resources (e.g., Wikipedia) can be employed. In this paper, we exploit two different sources of information to extract bilingual terminology from comparable corpora: the compositional and the contextual clue. The compositional clue is the hypothesis that the representations of a term in any pair of languages tend to consist of cor- responding lexical or sub-lexical units, e.g., pre- fixes, suffices and morphemes. In order to cap- ture associations of textual units across languages, we investigate three different character n-gram ap- proaches, namely a Random Forest (RF) classifier (Kontonatsios et al., 2014), Support Vector Ma- chines with an RBF kernel (SVM-RBF) and a Lo- gistic Regression (LogReg) classifier. Whilst the previous approaches take as an input monolingual features and then try to find cross-lingual map- pings, our proposed method (LogReg classifier) considers multilingual features, i.e., tuples of co- occurring n-grams. The contextual clue …

Content truncated to 500 words