Cross-Lingual Training of Neural Models for Document Ranking

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2768–2773 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2768 Cross-Lingual Training of Neural Models for Document Ranking Peng Shi, He Bai, and Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo {peng.shi, he.bai, jimmylin}@uwaterloo.ca Abstract We tackle the challenge of cross-lingual training of neural document ranking mod- els for mono-lingual retrieval, specifically leveraging relevance judgments in English to improve search in non-English languages. Our work successfully applies multi-lingual BERT (mBERT) to document ranking and ad- ditionally compares against a number of alter- natives: translating the training data, translat- ing documents, multi-stage hybrids, and en- sembles. Experiments on test collections in six different languages from diverse language fam- ilies reveal many interesting findings: model- based relevance transfer using mBERT can significantly improve search quality in (non- English) mono-lingual retrieval, but other “low resource” approaches are competitive as well. 1 Introduction This work proposes techniques for leveraging rel- evance judgments in a source language (English) to train neural models for mono-lingual document retrieval in multiple target (non-English) languages, what we refer to as cross-lingual training. Success in this task would make it easier to develop effec- tive search engines in multiple (potentially low- resource) languages, without gathering expensive relevance judgments in each language. A blog post by Google suggests that the company is exploring this approach to improving web search across a number of languages.1 We are inspired by the work of Wu and Dredze (2019), who explored the cross-lingual potential of multi-lingual BERT as a zero-shot language transfer model for NLP tasks such as named-entity recognition and parsing. Mono-lingual BERT mod- els (Devlin et al., 2019) have also proven effec- tive in document retrieval (Dai and Callan, 2019; 1https://www.blog. google/products/search/ search-language-understanding-bert/ MacAvaney et al., 2019; Li et al., 2020). In particu- lar, Akkalyoncu Yilmaz et al. (2019) demonstrated that BERT models fine-tuned with passage-level relevance data can transfer across domains: surpris- ingly, fine-tuning on social media data is effective for relevance classification on newswire documents without any additional modifications. Building on these results, we wondered if multi-lingual BERT could enable cross-lingual training of neural docu- ment ranking models as well. The contribution of this work is to explore di- verse methods to train neural document ranking models cross-lingually. While we are aware of two previous papers along these lines (Shi and Lin, 2019; MacAvaney et al., 2020), this work ex- plores a far broader range of techniques and adds more nuance to previous findings. Beyond the ba- sic approach proposed by these two papers, which we refer to as model-based transfer, we investi- gate additional approaches involving the translation of the training data, the translation of documents, hybrid models, as well as ensembles—which we broadly characterize into “high resource” and “low resource” settings. We show that various methods alone and in combination can yield robust increases in effectiveness across diverse languages with min- imal resources, and that model-based cross-lingual transfer isn’t the only way. 2 …

Content truncated to 500 words