Quantifying Language Variation Acoustically with Few Resources

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3735 - 3741 July 10-15, 2022 ©2022 Association for Computational Linguistics Quantifying Language Variation Acoustically with Few Resources Martijn Bartelds University of Groningen The Netherlands m.bartelds@rug.nl Martijn Wieling University of Groningen The Netherlands m.b.wieling@rug.nl Abstract Deep acoustic models represent linguistic infor- mation based on massive amounts of data. Un- fortunately, for regional languages and dialects such resources are mostly not available. How- ever, deep acoustic models might have learned linguistic information that transfers to low- resource languages. In this study, we evaluate whether this is the case through the task of dis- tinguishing low-resource (Dutch) regional vari- eties. By extracting embeddings from the hid- den layers of various wav2vec 2.0 models (including new models which are pre-trained and/or fine-tuned on Dutch) and using dynamic time warping, we compute pairwise pronun- ciation differences averaged over 10 words for over 100 individual dialects from four (re- gional) languages. We then cluster the result- ing difference matrix in four groups and com- pare these to a gold standard, and a partitioning on the basis of comparing phonetic transcrip- tions. Our results show that acoustic models outperform the (traditional) transcription-based approach without requiring phonetic transcrip- tions, with the best performance achieved by the multilingual XLSR-53 model fine-tuned on Dutch. On the basis of only six seconds of speech, the resulting clustering closely matches the gold standard. 1 Introduction Deep acoustic models have improved automatic speech recognition (ASR) substantially in recent years (Schneider et al., 2019; Baevski et al., 2020a,b; Conneau et al., 2020). These models represent linguistic information based on massive amounts of data. While these models are gen- erally evaluated on ASR benchmarks, few stud- ies have addressed what kind of linguistic infor- mation is represented by them. The work of Pasad et al. (2021) examined information repre- sented by the wav2vec 2.0 model (Baevski et al., 2020b) across the various Transformer layers. They showed that different layers encode different types of linguistic information. Specifically, the initial layers appeared to be most similar to the input speech features, whereas the middle layers mostly encoded contextual information. The final layers again turned out to be similar to the input speech features. However, the representations of the final layers changed when the model was fine- tuned, likely because task-specific information was learned. In addition, Ma et al. (2021) investigated several deep acoustic models using phonetic prob- ing tasks, and found that representations from these models capture information useful for distinguish- ing English phones. Importantly, these deep acous- tic models were better able to distinguish English phones than using conventional MFCC or filterbank features. Although they evaluated the transferabil- ity of deep acoustic representations across several domains, it remains unclear whether these mod- els learned information that transfers to other lan- guages. This is, however, important when working on more inclusive speech technology. Especially when resources for training these models are lack- ing, such as for …

Content truncated to 500 words