Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6471–6479 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6471 Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation Idris Abdulmumin1,6, Satya Ranjan Dash2, Musa Abdullahi Dawud2, Shantipriya Parida3, Shamsuddeen Hassan Muhammad4,5, Ibrahim Sa’id Ahmad6, Subhadarshi Panda7, Ondˇrej Bojar8, Bashir Shehu Galadanci6, Bello Shehu Bello6 1Department of Computer Science, Ahmadu Bello University, Zaria, Nigeria 2School of Computer Applications, KIIT University, Bhubaneswar, India 3Silo AI, Helsinki, Finland 4 LIAAD - INESC TEC, 5Faculty of Sciences-University of Porto, Portugal 6Faculty of Computer Science and Information Technology, Bayero University, Kano, Nigeria 7Graduate Center, City University of New York, USA 8Charles University, Faculty of Mathematics and Physics, ´UFAL, Prague, Czech Republic iabdulmumin@abu.edu.ng, sdashfca@kiit.ac.in, dawudmusa46@gmail.com, shantipriya.parida@silo.ai, {shmuhammad.csc, isahmad.it, bsgaladanci.se, bsbello.cs}@buk.edu.ng, spanda@gradcenter.cuny.edu, bojar@ufal.mff.cuni.cz Abstract Multi-modal Machine Translation (MMT) enables the use of visual information to enhance the quality of translations. The visual information can serve as a valuable piece of context information to decrease the ambiguity of input sentences. Despite the increasing popularity of such a technique, good and sizeable datasets are scarce, limiting the full extent of their potential. Hausa, a Chadic language, is a member of the Afro-Asiatic language family. It is estimated that about 100 to 150 million people speak the language, with more than 80 million indigenous speakers. This is more than any of the other Chadic languages. Despite a large number of speakers, the Hausa language is considered low-resource in natural language processing (NLP). This is due to the absence of sufficient resources to implement most NLP tasks. While some datasets exist, they are either scarce, machine-generated, or in the religious domain. Therefore, there is a need to create training and evaluation data for implementing machine learning tasks and bridging the research gap in the language. This work presents the Hausa Visual Genome (HaVG), a dataset that contains the description of an image or a section within the image in Hausa and its equivalent in English. To prepare the dataset, we started by translating the English description of the images in the Hindi Visual Genome (HVG) into Hausa automatically. Afterward, the synthetic Hausa data was carefully post-edited considering the respective images. The dataset comprises 32,923 images and their descriptions that are divided into training, development, test, and challenge test set. The Hausa Visual Genome is the first dataset of its kind and can be used for Hausa-English machine translation, multi-modal research, and image description, among various other natural language processing and generation tasks. Keywords: Natural Language Processing, Machine Translation, Hausa Language Resources, Low Resource Languages, Multi- modal dataset, Visual Genome 1. Introduction Machine translation is the use of a computer to automat- ically generate the equivalent of a given source text in a language that is different from the original language. While Neural Machine Translation (NMT) (Bahdanau et al., 2015; Vaswani et al., 2017; Gehring et al., 2017) has revolutionized automatic translation, the absence of sufficient training data in many …

Content truncated to 500 words