AfriCLIRMatrix: Enabling Cross-Lingual Information Retrieval for African Languages

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8721 - 8728 December 7-11, 2022 ©2022 Association for Computational Linguistics AfriCLIRMatrix: Enabling Cross-Lingual Information Retrieval for African Languages Odunayo Ogundepo1, Xinyu Zhang1, Shuo Sun2, Kevin Duh2, and Jimmy Lin1 1David R. Cheriton School of Computer Science, University of Waterloo 2 John Hopkins University 1{oogundep, xinyucrystina.zhang, jimmylin}@uwaterloo.ca 2{ssun32@jhu.edu, kevinduh@cs.jhu.edu} Abstract Language diversity in NLP is critical in en- abling the development of tools for a wide range of users. However, there are limited re- sources for building such tools for many lan- guages, particularly those spoken in Africa. For search, most existing datasets feature few or no African languages, directly impacting re- searchers’ ability to build and improve infor- mation access capabilities in those languages. Motivated by this, we created AfriCLIRMatrix, a test collection for cross-lingual information retrieval research in 15 diverse African lan- guages. In total, our dataset contains 6 mil- lion queries in English and 23 million rele- vance judgments automatically mined from Wikipedia inter-language links, covering many more African languages than any existing in- formation retrieval test collection. In addition, we release BM25, dense retrieval, and sparse– dense hybrid baselines to provide a starting point for the development of future systems. We hope that these efforts can spur additional work in search for African languages. Afri- CLIRMatrix can be downloaded at https:// github.com/castorini/africlirmatrix. 1 Introduction The ever-increasing amounts of information on the web in different languages highlight the need for systems that enable users to search in one language and retrieve relevant documents in another. This search task, commonly known as cross-lingual in- formation retrieval (CLIR), is becoming increas- ingly important. CLIR can break down language barriers between information seekers and the exten- sive collections of documents that are available in diverse languages. One common approach to CLIR takes advan- tage of machine translation and monolingual in- formation retrieval (Zhou et al., 2012; Jiang et al., 2020). The documents and queries are translated into the same language before search occurs. This translation is often performed using a variety of sources, including parallel corpora, bilingual dic- tionaries, and machine translation (MT) systems. The effectiveness of this approach relies heavily on translation quality, which may be a bottleneck for low-resource languages where high-quality transla- tions are not readily available. To address this challenge, researchers have re- cently explored the use of pretrained multilingual models (MacAvaney et al., 2020; Shi et al., 2020). Examples such as mBERT (Devlin et al., 2019) and XLM-R (Conneau et al., 2020) are often pretrained on a large collection of multilingual texts, enabling the models to learn representations across differ- ent languages. The use of multilingual models for CLIR often builds on techniques that have previ- ously been applied to monolingual retrieval (Lin et al., 2021b). Regardless of approach, modern neural-based CLIR models are data hungry, typically requiring large amounts of query–document pairs that have been annotated with relevance labels. Such anno- tated data are expensive to obtain, especially for low-resource African …

Content truncated to 500 words