BIG-C: a Multimodal Multi-Purpose Dataset for Bemba

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2062–2078 July 9-14, 2023 ©2023 Association for Computational Linguistics BIG-C: a Multimodal Multi-Purpose Dataset for Bemba Claytone Sikasote1, Eunice Mukonde2, Md Mahfuz Ibn Alam3, Antonios Anastasopoulos3 1Department of Computer Science, University of Zambia, Zambia 2Department of Literature and Languages, University of Zambia, Zambia 3Department of Computer Science, George Mason University, USA claytone.sikasote@cs.unza.zm, antonis@gmu.edu Abstract We present BIG-C (Bemba Image Grounded Conversations), a large multimodal dataset for Bemba. While Bemba is the most populous language of Zambia, it exhibits a dearth of resources which render the development of language technologies or language process- ing research almost impossible. The dataset is comprised of multi-turn dialogues between Bemba speakers based on images, transcribed and translated into English. There are more than 92,000 utterances/sentences, amounting to more than 180 hours of audio data with cor- responding transcriptions and English transla- tions. We also provide baselines on speech recognition (ASR), machine translation (MT) and speech translation (ST) tasks, and sketch out other potential future multimodal uses of our dataset. We hope that by making the dataset available to the research community,1 this work will foster research and encourage collabora- tion across the language, speech, and vision communities especially for languages outside the "traditionally" used high-resourced ones. 1 Introduction The Bemba language, spoken by over 10 million people in Zambia and other parts of Africa, is a rich and vibrant language with a unique cultural heritage. However, despite its significance, Bemba is a dramatically under-resourced language, lacking in high-quality language data and resources for natural language processing (NLP) experiments and for the development of language technologies. With this work, we address this issue by creating a new multimodal dataset for Bemba. Our goal is to improve the accuracy and effectiveness of NLP systems for speakers of Bemba and support research in this under-served language. While most datasets are constructed with a spe- cific task in mind and tailored to its characteris- 1All data and code are publicly available: https:// github.com/csikasote/bigc. Umwana alepeluka pa cimpelwa elo ba nyina pa mbali balemutamba A child is on a swing chair swinging while the mother watches on the side Akamwana kali akasansamuka saana,kalelosha ba nyina nakasekelela. The child is very excited for swinging, looking at its mother with a smile. Cipalile pali efyo balekeba ba nyina pantu bonse bali abasekelea It seems like she is telling the child something as they both have smiled. Limbi kaleumfwafye bwino filyafine kalepeluka nabanyina balekasunka. It may be excited for swinging even as the mother swings it. Awe cine abana balatemwa banyina, insansa muli uyu umwana shafulisha Indeed children love their mothers, the child is just too excited. Figure 1: Example of the data included in BIG-C. The grounding image (top) and the ensuing Bemba dialogue transcribed and translated in English. tics, we aim to provide a path towards building multi-purpose datasets. Under a limited budget, we hypothesize that the ideal scenario is to create datasets that …

Content truncated to 500 words