Exploiting Bilingual Information to Improve Web Search

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 1075–1083, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP Exploiting Bilingual Information to Improve Web Search Wei Gao1, John Blitzer2, Ming Zhou3, and Kam-Fai Wong1 1The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, China {wgao,kfwong}@se.cuhk.edu.hk 2Computer Science Division, University of California at Berkeley, CA 94720-1776, USA blitzer@cs.berkeley.edu 3Microsoft Research Asia, Beijing 100190, China mingzhou@microsoft.com Abstract Web search quality can vary widely across languages, even for the same information need. We propose to exploit this variation in quality by learning a ranking function on bilingual queries: queries that appear in query logs for two languages but represent equivalent search interests. For a given bilingual query, along with correspond- ing monolingual query log and monolin- gual ranking, we generate a ranking on pairs of documents, one from each lan- guage. Then we learn a linear ranking function which exploits bilingual features on pairs of documents, as well as standard monolingual features. Finally, we show how to reconstruct monolingual ranking from a learned bilingual ranking. Us- ing publicly available Chinese and English query logs, we demonstrate for both lan- guages that our ranking technique exploit- ing bilingual data leads to significant im- provements over a state-of-the-art mono- lingual ranking algorithm. 1 Introduction Web search quality can vary widely across lan- guages, even for a single query and search en- gine. For example, we might expect that rank- ing search results for the query ��� ��� (Thomas Hobbes) to be more difficult in Chinese than it is in English, even while holding the ba- sic ranking function constant. At the same time, ranking search results for the query Han Feizi (� ��) is likely to be harder in English than in Chi- nese. A large portion of web queries have such properties that they are originated in a language different from the one they are searched. This variance in problem difficulty across lan- guages is not unique to web search; it appears in a wide range of natural language processing prob- lems. Much recent work on bilingual data has fo- cused on exploiting these variations in difficulty to improve a variety of monolingual tasks, includ- ing parsing (Hwa et al., 2005; Smith and Smith, 2004; Burkett and Klein, 2008; Snyder and Barzi- lay, 2008), named entity recognition (Chang et al., 2009), and topic clustering (Wu and Oard, 2008). In this work, we exploit a similar intuition to im- prove monolingual web search. Our problem setting differs from cross-lingual web search, where the goal is to return machine- translated results from one language in response to a query from another (Lavrenko et al., 2002). We operate under the assumption that for many mono- lingual English queries (e.g., Han Feizi), there ex- ist good documents in English. If we have Chinese information as well, we can exploit it to help find these documents. As we will see, machine trans- lation can provide important predictive informa- tion …

Content truncated to 500 words