Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8608 - 8621 December 7-11, 2022 ©2022 Association for Computational Linguistics Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks Colin Leong*, Joshua Nemecek†, Jacob Mansdorfer§, Anna Filighera¶, Abraham Owodunni|| and Daniel Whitenack† *University of Dayton Research Institute,†SIL International,§Independent Contractor, ¶TU Darmstadt and ||Masakhane cleong1@udayton.edu, joshua_nemecek@sil.org jacob.mansdorfer@gmail.com, anna.filighera@kom.tu-darmstadt.de owodunniabraham@gmail.com, dan_whitenack@sil.org Abstract We present Bloom Library, a linguistically diverse set of multimodal and multilingual datasets for language modeling, image cap- tioning, visual storytelling, and speech synthe- sis/recognition. These datasets represent ei- ther the most, or among the most, multilingual datasets for each of the included downstream tasks. In total, the initial release of the Bloom Library datasets covers 363 languages across 32 language families. We train downstream task models for various languages represented in the data, showing the viability of the data for future work in low-resource, multimodal NLP and establishing the first known baselines for these downstream tasks in certain languages (e.g., Bisu [bzi], with an estimated population of 700 users). Some of these first-of-their-kind baselines are comparable to state-of-the-art per- formance for higher-resourced languages. The Bloom Library datasets are released under Cre- ative Commons licenses on the Hugging Face datasets hub to catalyze more linguistically diverse research in the included downstream tasks. 1 Introduction Only a negligible fraction of the 7100+ living languages (Eberhard et al., 2021) have sufficient, publicly available text, audio, and image data to train state-of-the-art language/speech models and/or models for downstream tasks like Named Entity Recognition (NER) or image captioning. This data scarcity results in systematic inequali- ties in the performance of NLP tasks across the world’s languages (Blasi et al., 2021). Indigenous language ecologies also represent profoundly dif- ferent understandings of the nature and function of language (Bird, 2022, 2020), which might prior- itize orality or translanguaging (Quakenbush and Simons, 2018), for example, above a single, written mode of communication in all domains. The Bloom Library1 is a web-based platform that is attempting to facilitate an increase in the amount of multimodal materials available to communities speaking non-dominant languages. The Bloom Li- brary holds over 12,400 books in 545 languages (at the time this paper is published), covering sub- jects including agriculture, business, culture, math, science, religion, and health. Many of these books include images aligned with text, and 1,600+ of the books have corresponding audio recordings (called "talking books"). Language communities can cre- ate new books, create audio recordings, download existing books, and translate existing books using the open-source "Bloom" software2. To boost language diversity and indigenous per- spectives in the NLP research community, we present multimodal datasets post-processed out of the Bloom Library. We anticipate that more task-specific datasets will be created from the Bloom Library. However, as a starting point, we are presenting the following datasets: (1) bloom-lm for language modeling in 351 lan- guages; (2) bloom-captioning for image-to- text or text-to-image tasks in 351 languages; (3) bloom-vist for visual storytelling in 351 lan- …

Content truncated to 500 words