Query translation using Wikipedia-based resources for analysis and disambiguation

Query Translation using Wikipedia-based resources for analysis and disambiguation Benoît Gaillard Orange Labs & CLLE-ERSS, 5, allées A. Machado, F-31058 Toulouse Cedex 9, France benoit.gaillard @univ-tlse2.fr Malek Boualem Orange Labs 2, Av. Pierre Marzin 22300 Lannion, France malek.boualem @orange-ftgroup.com Olivier Collin Orange Labs 2, Av. Pierre Marzin 22300 Lannion, France olivier.collin @orange-ftgroup.com Abstract This work investigates query translation using only Wikipedia-based resources in a two step approach: analysis and disam- biguation. After arguing that data mined from Wikipedia is particularly relevant to query translation, both from a lexical and a semantic perspective, we detail the im- plementation of the approach. In the analysis phase, lexical units are extracted from queries and associated to several possible translations using a Wikipedia- based bilingual dictionary. During the second phase, one translation is chosen amongst the many candidates, based on topic homogeneity, asserted with the help of semantic information carried by cate- gories of Wikipedia articles. We report promising results regarding translation accuracy. 1 Introduction Retrieving relevant information from the con- stantly increasing amounts of available multilin- gual content on the web is becoming as signifi- cant an issue as providing access to content originally was. To address this issue, Cross Lan- guage Information Retrieval (CLIR) techniques are used to enable users to retrieve relevant documents in a language different from the lan- guage of queries. To compare a query in a lan- guage to documents in another language, CLIR systems often apply Machine Translation (MT) techniques either to queries or to all indexed documents. Comparative evaluations (Clough, 2005) suggest that translating documents before © 2010 European Association for Machine Translation. indexing them is a slightly better approach to CLIR than translating queries because transla- tions of indexed data tend to be more accurate than translations of queries. This approach has been occasionally studied, for example in (Gail- lard et al, 2010), but major CLIR systems seem to favor query translation approaches because translating large indexes into many languages would be too costly. The most direct approach to translate queries is probably to use multilingual dictionaries, as for example in the prototype1 de- tailed in (Etzioni et al, 2007). Two major diffi- culties face the lexical approach. First, the cover- age of lexicons is a limiting factor that is difficult and expensive to optimize especially because queries can refer to a great number of named en- tities and multi-word terms the list of which is constantly and rapidly growing. Secondly, most words have multiple meanings. Selecting the most appropriate translation between several al- ternatives is a crucial yet challenging task, as queries often provide very little information that can be used to disambiguate. Wikipedia has features that can provide solu- tions to these two issues. Thanks to the voluntary contributions of millions of users, it gathers a very significant amount of continuously updated, freely accessible organized knowledge. From it, one can easily extract up to date multilingual dic- tionaries that have an optimal lexical coverage. Furthermore, Wikipedia content is classified in a hierarchical network …

Content truncated to 500 words