Generating Monolingual Dataset for Low Resource Language Bodo from old books using Google Keep

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6563–6570 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6563 Generating Monolingual Dataset for Low Resource Language Bodo from old books using Google Keep Sanjib Narzarya, Maharaj Brahmab, Mwnthai Narzaryc, Gwmsrang Muchaharyd Pranav Kumar Singhe, Apurbalal Senapatif, Sukumar Nandig, Bidisha Somh a,b,c,d,e,f - Department of Computer Science and Engineering, Central Institute of Technology Kokrajhar Kokrajhar, India g,h - Center for Linguistic Science & Technology Indian Institute of Technology Guwahati Guwahati, India {san, p20cse1012, p20cse1001, p20cse1011, p.singh, a.senapati}@cit.ac.in {sukumar, bidishar}@iitg.ac.in Abstract Bodo is a scheduled Indian language spoken largely by the Bodo community of Assam and other northeastern Indian states. Due to a lack of resources, it is difficult for young languages to communicate more effectively with the rest of the world. This leads to a lack of research in low-resource languages. The creation of a dataset is a tedious and costly process, particularly for languages with no participatory research. This is more visible for languages that are young and have recently adopted standard writing scripts. In this paper, we present a methodology using Google Keep for OCR to generate a monolingual Bodo corpus from different books. In this work, a Bodo text corpus of 192,327 tokens and 32,268 unique tokens is generated using free, accessible, and daily-usable applications. Moreover, some essential characteristics of the Bodo language are discussed that are neglected by Natural Language Processing (NLP) researchers. Keywords: Monolingual Corpus Creation, Low Resource Language, Bodo Language, Vulnerable Language 1. Introduction Bodo is one of the Indic languages and belongs to the Sino-Tibetan language family, one of the four language families widely spoken in India. Ac- cording to the 2011 Census (Census, 2011a)1, it has nearly a million speakers. It is primarily spo- ken by the Bodo tribe in Assam, as well as tribes such as the Kachari, Mech, and others. There are 1,454,547 native Bodo speakers and total of 1,482,929 Bodo speakers (Census, 2011b) as shown in Table 1. It accounts for 0.12 % of India’s overall population and ranks at 21 out of other 22 sched- uled languages as shown in Table 2. As in Figure 1, the number of Bodo speakers is rising steadily. Historically, Bodo has a rich oral tradition but no standard script for writing, and until recently, the Devanagari script is officially adopted. The Natu- ral Language Processing (NLP) literature on Bodo language is relatively small, and the majority of research has been carried out just recently. This can be attributed to the following reasons: (a) the youngness of the language, (b) low data availabil- ity, (c) low NLP research interest, (d) unavailabil- ity of preliminary studies and benchmarks, and (e) lack of technical resources. NLP is an emerging field of study with rising ap- plications in various domains. There is a huge de- 1Survey conducted by Government of India 0.5 M 1 M 1.5 M 2 M 2011 2001 1991 1971 1,482,929 1,350,478 1,221,881 556,576 # …

Content truncated to 500 words