Speech Synthesis of Code-Mixed Text

Speech Synthesis of Code-Mixed Text Sunayana Sitaram and Alan W Black Carnegie Mellon University Pittsburgh, USA ssitaram, awb@cs.cmu.edu Abstract Most Text to Speech (TTS) systems today assume that the input text is in a single language and is written in the same language that the text needs to be synthesized in. However, in bilingual and multilingual communities, code mixing or code switching occurs in speech, in which speakers switch between languages in the same utterance. Due to the popularity of social media, we now see code-mixing even in text in these multilingual communities. TTS systems capable of synthesizing such text need to be able to handle text that is written in multiple languages and scripts. Code-mixed text poses many challenges to TTS systems, such as language identification, spelling normalization and pronunciation modeling. In this work, we describe a preliminary framework for synthesizing code-mixed text. We carry out experiments on synthesizing code-mixed Hindi and English text. We find that there is a significant user preference for TTS systems that can correctly identify and pronounce words in different languages. Keywords: speech synthesis, code-mixing, multilingual systems 1. Introduction Text-to-Speech (TTS) systems synthesize speech from text, and this text is usually assumed to be written in a standard- ized written form used for the target language. However, in many cases, a language is written using the script of an- other language, even if it has its own standardized written form. A common phenomenon that occurs in text in Social Media, Instant Messaging and email in bilingual and mul- tilingual societies is code-switching, in which users type multiple languages in the same script, or sometimes retain the original script of the languages that are being mixed in the sentence. Code switching is defined as switching between different languages in speech or text, in which the grammatical forms of the languages are preserved. Typically, one can iden- tify one, or a few points in a sentence where code switch- ing occurs. Code-mixing refers to the mixing of phrases, words and morphemes of one language into another lan- guage (Myers-Scotton, 1997). Lexical borrowing occurs when a word from one language is borrowed and used in another language while following the syntactic rules of the borrowing language. Code switching can be thought of as an an inter-sentential phenomenon while code-mixing can be thought of as an intra-sentential phenomenon, although the techniques we describe in this paper are agnostic to the difference be- tween the two. Code-mixing can also occur at the intra- word level, which we don’t explicitly address in this work. From this point onwards, we choose to ignore the difference between code switching and code-mixing and use the term code-mixing as a general term to denote mixed languages in text. Code-mixed text provides multiple challenges to TTS sys- tems: first, the system needs to identify what languages are being mixed in the sentence, and then identify the language of each word or morpheme. Secondly, since languages be- ing mixed may not have standardized spellings particularly if …

Content truncated to 500 words