Boosting Cross-Language Retrieval by Learning Bilingual Phrase Associations from Relevance Rankings

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1688–1699, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Boosting Cross-Language Retrieval by Learning Bilingual Phrase Associations from Relevance Rankings Artem Sokolov and Laura Jehl and Felix Hieber and Stefan Riezler Department of Computational Linguistics Heidelberg University, 69120 Heidelberg, Germany {sokolov,jehl,hieber,riezler}@cl.uni-heidelberg.de Abstract We present an approach to learning bilin- gual n-gram correspondences from relevance rankings of English documents for Japanese queries. We show that directly optimizing cross-lingual rankings rivals and complements machine translation-based cross-language in- formation retrieval (CLIR). We propose an ef- ficient boosting algorithm that deals with very large cross-product spaces of word correspon- dences. We show in an experimental evalu- ation on patent prior art search that our ap- proach, and in particular a consensus-based combination of boosting and translation-based approaches, yields substantial improvements in CLIR performance. Our training and test data are made publicly available. 1 Introduction The central problem addressed in Cross-Language Information Retrieval (CLIR) is that of translating or projecting a query into the language of the docu- ment repository across which retrieval is performed. There are two main approaches to tackle this prob- lem: The first approach leverages the standard Sta- tistical Machine Translation (SMT) machinery to produce a single best translation that is used as search query in the target language. We will hence- forth call this the direct translation approach. This technique is particularly useful if large amounts of data are available in domain-specific form. Alternative approaches avoid to solve the hard problem of word reordering, and instead rely on token-to-token translations that are used to project the query terms into the target language with a probabilistic weighting of the standard term tf-idf scheme. Darwish and Oard (2003) termed this method the probabilistic structured query approach. The advantage of this technique is an implicit query expansion effect due to the use of probability distri- butions over term translations (Xu et al., 2001). Re- cent research has shown that leveraging query con- text by extracting term translation probabilities from n-best direct translations of queries instead of using context-free translation tables outperforms both di- rect translation and context-free projection (Ture et al., 2012b; Ture et al., 2012a). While direct translation as well as probabilistic structured query approaches use machine learning to optimize the SMT module, retrieval is done by stan- dard search algorithms in both approaches. For ex- ample, Google’s CLIR approach uses their standard proprietary search engine (Chin et al., 2008). Ture et al. (2012b; 2012a) use standard retrieval algorithms such as BM25 (Robertson et al., 1998). That means, machine learning in SMT-based approaches concen- trates on the cross-language aspect of CLIR and is agnostic of the ultimate ranking task. In this paper, we present a method to project search queries into the target language that is com- plementary to SMT-based CLIR approaches. Our method learns a table of n-gram correspondences by direct optimization of a ranking objective on rele- vance rankings of English documents for Japanese …

Content truncated to 500 words