Preparation of Bangla Speech Corpus from Publicly Available Audio & Text

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6586–6592 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6586 Preparation of Bangla Speech Corpus from Publicly Available Audio & Text Shafayat Ahmed∗†, Nafis Sadeq∗†, Sudipta Saha Shubha∗†, Md. Nahidul Islam∗ Muhammad Abdullah Adnan∗, Mohammad Zuberul Islam‡ ∗Bangladesh University of Engineering and Technology (BUET) shafayat, nafis, sudipta, nahid.rimon@ra.cse.buet.ac.bd, adnan@cse.buet.ac.bd ‡Samsung R & D Institute Bangladesh m.zuberul@samsung.com Abstract Automatic speech recognition systems require large annotated speech corpus. Manual annotation of a large corpus is very difficult. In this paper, we focus on automatic preparation of a speech corpus for Bangladeshi Bangla. We have used publicly available Bangla audiobooks and TV news recordings as audio sources. We designed and implemented an iterative algorithm that takes as input a speech corpus and a huge amount of raw audio (without transcription) and outputs a much larger speech corpus with reasonable confidence. We have leveraged speaker diarization, gender detection, etc. to prepare the annotated corpus. We also have prepared a synthetic speech corpus for handling out-of-vocabulary word problem in Bangla language. Our corpus is suitable for training with Kaldi. Experimental results show that use of our corpus in addition to Google Speech corpus (229 hours) significantly improves the performance of the ASR system. Keywords: Speech recognition, Speech corpus generation 1. Introduction Automatic Speech Recognition (ASR) systems require an annotated speech corpus for training. Speech corpus refers to a collection of audio files with corresponding text tran- scriptions. The duration of each speech segment is kept under 35 seconds (Panayotov et al., 2015). The speech corpus also contains some additional information such as the gender of speakers, recording environment, etc. For achieving WER of less than 15% for speech recognition, we need a huge annotated speech corpus, more than 1000 hours according to (Moore, 2003). Very few publicly available speech datasets are available for Bangladeshi Bangla. The largest publicly available corpus is the 229 hours speech corpus released by Google (Kjartansson et al., 2018). Different approaches for speech corpus development have been explored by the researchers. A basic approach is to manually transcribe existing audio files. This is a very time consuming, monotonous, and error-prone task. Tran- scription of one-hour recording can take 3 to 5 hours or more (Hazen, 2006). A comparatively faster approach is to develop an interactive mobile application that prompts the user to read a particular text (Hughes et al., 2010). The user will have to start recording manually, read the prompted text, and end recording manually. In our expe- rience, preparing one hour’s worth of transcribed speech takes around 2 hours in this approach. But this approach still requires some manual labor. A large number of speak- ers need to be motivated to use this application. It is not cost-effective to offer an attractive incentive for every speaker participating in this process. Some works such as (Panayotov et al., 2015) used forced alignment techniques to produce speech corpus from public domain audiobooks and …

Content truncated to 500 words