Cross-Lingual Speaker Identification for Indian Languages

Proceedings of Recent Advances in Natural Language Processing, pages 979–987 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_105 979 Cross-Lingual Speaker Identification for Indian Languages Amaan Rizvi Anupam Jamatia Dwijen Rudrapal Kunal Chakma Department of Computer Science and Engineering National Institute of Technology Agartala Tripura, India {amaan.rizvi39,anupamjamatia,dwijen.rudrapal,kchax4377}@gmail.com Björn Gambäck Department of Computer Science Norwegian University of Science and Technology Trondheim, Norway gamback@ntnu.no Abstract The paper introduces a cross-lingual speaker identification system for Indian languages, util- ising a Long Short-Term Memory dense neu- ral network (LSTM-DNN). The system was trained on audio recordings in English and eval- uated on data from Hindi, Kannada, Malay- alam, Tamil, and Telugu, with a view to how factors such as phonetic similarity and native accent affect performance. The model was fed with MFCC (mel-frequency cepstral co- efficient) features extracted from the audio file. For comparison, the corresponding mel- spectrogram images were also used as input to a ResNet-50 model, while the raw audio was used to train a Siamese network. The LSTM-DNN model outperformed the other two models as well as two more traditional base- line speaker identification models, showing that deep learning models are superior to probabilis- tic models for capturing low-level speech fea- tures and learning speaker characteristics. 1 Introduction Ascertaining the identities of the writers and speak- ers are important tasks in language and speech pro- cessing. The vocabulary a person uses as well as the ways a person writes and talks can give us information about their identity or their back- ground. Furthermore, people’s voices are unique identifiers, just like their retinas and fingerprints, making speaker recognition (the task of recognis- ing the voice of a speaker based on audio input) applicable to building human-to-machine interac- tion and biometric solutions such as voice assis- tants, voice-controlled services, and speech-based authentication products (Beigi, 2011). There are two basic speaker recognition tasks: (i) Speaker Verification: confirm the identity of a speaker. (ii) Speaker Identification: identify a voice in a set of speakers. Speaker recognition can be monolingual as well as cross-lingual (Sale et al., 2018). For monolingual tasks, the same language is used to both train and test models. In cross-lingual speaker recognition, a model is trained on one language, e.g., English, and tested on a different language, e.g., Arabic. In a multilingual country like India, with more than 120 languages having tens of thousands of speakers and some 50 languages having official sta- tus at national or regional level, most citizens speak several languages fluently. Due to this plethora of multilingual speakers, it is not feasible to train a speaker recognition model in one language and re-train the model in a new language. Therefore, the development of cross-language speaker recog- nition models has become a salient task. Intu- itively, language mismatch in training and test lan- guage should not be a problem, since a person’s vocal traits have nothing to do with what they are saying, but in general, the performance of a speaker recognition system still degrades when a model is trained on one …

Content truncated to 500 words