Improving Code-switched ASR with Linguistic Information

Proceedings of the 29th International Conference on Computational Linguistics, pages 7171–7176 October 12–17, 2022. 7171 Improving code-switched ASR with linguistic information Jie Chi and Peter Bell Centre for Speech Technology Research, University of Edinburgh, UK Abstract This paper seeks to improve the performance of automatic speech recognition (ASR) sys- tems operating on code-switched speech. Code- switching refers to the alternation of languages within a conversation, a phenomenon that is of increasing importance considering the rapid rise in the number of bilingual speakers in the world. It is particularly challenging for ASR owing to the relative scarcity of code-switching speech and text data, even when the individual languages are themselves well-resourced. This paper proposes to overcome this challenge by applying linguistic theories in order to gener- ate more realistic code-switching text, neces- sary for language modelling in ASR. Working with English-Spanish code-switching, we find that Equivalence Constraint theory and part-of- speech labelling are particularly helpful for text generation, and bring 2% improvement to ASR performance. 1 Introduction Although accurate statistics on the number of worldwide bilingual speakers are hard to determine, it has been generally believed that more than half of the population can communicate in more than one language (Ansaldo et al., 2008; Bialystok et al., 2012; Grosjean, 2010). With the rising popularity of voice assistant and translation applications, auto- matic speech recognition (ASR) systems have been increasingly integrated into people’s lives. Consid- ering the abundance of bilingual countries 1, and code-switching 2 is common in everyday conversa- tions, there has been great interest into developing such system in the corresponding setting. The most widely accepted definition of code- switching is the phenomenon whereby a speaker 1https://www.uottawa.ca/clmc/55-bilingual-countries- world 2Code-switching can also be found in text, such as social media or news paper headlines, but in this paper we are only focus only on the spoken form shifts from one language to another within a single utterance, especially in an environment in which both languages are being used (Heredia and Al- tarriba, 2001). Some previous work make a dis- tinction between code-switching and code-mixing, where the former occurs at sentence-level while the latter occurs at word-level (Myers-Scotton, 1997; Gumperz, 1982). However, in recent years, this dis- tinction has becomes unclear (Bali et al., 2014). To avoid confusion, we will only use the term code-switching in this work, and denote the dif- ferences in switching position as sub-types of it (Myers-Scotton, 1989; Muysken et al., 2000; Ma- jor, 2002; Winford, 2003). Although different lan- guage pairs may present varying extents or types of code-switching, they can generally be categorised as inter-sentential, intra-sentential, and tag switch- ing, where respectively the language switches at sentence or clause boundary; within the sentence or clause; or by inserting a tag phrase 3. In this pa- per, we focus only intra-sentential switching, which is a much harder task compared with other types, because the acoustic variance of mixed languages within the sentence can be larger than across sen- tence (Li et al., 2019). The challenge of developing …

Content truncated to 500 words