DITTO: Data-efficient and Fair Targeted Subset Selection for ASR Accent Adaptation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5810–5822 July 9-14, 2023 ©2023 Association for Computational Linguistics DITTO: Data-efficient and Fair Targeted Subset Selection for ASR Accent Adaptation Suraj Kothawade1∗, Anmol Mekala2∗, Chandra Sekhara D2, Mayank Kothyari2, Rishabh Iyer1, Ganesh Ramakrishnan2, Preethi Jyothi2 1 The University of Texas at Dallas, Dallas, USA 2 Indian Institute of Technology Bombay, Mumbai, India Abstract State-of-the-art Automatic Speech Recognition (ASR) systems are known to exhibit disparate performance on varying speech accents. To im- prove performance on a specific target accent, a commonly adopted solution is to finetune the ASR model using accent-specific labeled speech. However, acquiring large amounts of labeled speech for specific target accents is challenging. Choosing an informative subset of speech samples that are most representative of the target accents becomes important for effective ASR finetuning. To address this prob- lem, we propose DITTO (Data-efficient and faIr Targeted subseT selectiOn) that uses Submod- ular Mutual Information (SMI) functions as ac- quisition functions to find the most informa- tive set of utterances matching a target accent within a fixed budget. An important feature of DITTO is that it supports fair targeting for multiple accents, i.e. it can automatically se- lect representative data points from multiple accents when the ASR model needs to perform well on more than one accent. We show that DITTO is 3-5 times more label-efficient than other speech selection methods on the Indic- TTS and L2 datasets. 1 Introduction State-of-the-art speech recognition systems have seen tremendous progress in the last few years, with end-to-end architectures becoming a default mod- eling choice. While end-to-end models yield im- pressive Word Error Rates (WERs) and work well for certain user populations (Rao et al., 2017; Chiu et al., 2018), they severely underperform when con- fronted with out-of-domain test utterances in target accents that are unseen or rarely seen during train- ing (Feng et al., 2021; Koenecke et al., 2020). A common solution (Shor et al., 2019; Sim et al., 2019) to address such mismatched settings is to adapt a well-trained, speaker-independent ASR ∗Equal contribution. model with a small amount of accent-specific target data to adapt models to the target setting. While these works propose different fine-tuning sched- ules that would be most beneficial given the limited amount of target data, the question of which utter- ances should be chosen in order to be transcribed and further used for fine-tuning has received far less attention. This is extremely important, since procuring and labeling accent-specific data is chal- lenging and expensive. Awasthi et. al. (Awasthi et al., 2021) present a method to select sentences within a fixed budget that are most likely to induce ASR errors to record accented audio on, resulting in higher-quality personalized ASR models for target accents compared to random selection. However, they assume access to a small seed set of labeled utterances from the target speaker. We address a more realistic setting wherein we have access only to a limited number of …

Content truncated to 500 words