ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7259–7268 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7259 ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation Holy Lovenia⋆, Samuel Cahyawijaya⋆, Genta Indra Winata⋆†, Peng Xu, Xu Yan, Zihan Liu, Rita Frieske, Tiezheng Yu, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram E. Shi, Pascale Fung The Hong Kong University of Science and Technology {hlovenia, scahyawijaya, giwinata}@connect.ust.hk Abstract Code-switching is a speech phenomenon occurring when a speaker switches language during a conversation. Despite the spontaneous nature of code-switching in conversational spoken language, most existing works collect code-switching data from read speech instead of spontaneous speech. ASCEND (A Spontaneous Chinese-English Dataset) is a high-quality Mandarin Chinese-English code-switching corpus built on spontaneous multi-turn conversational dialogue sources collected in Hong Kong. We report ASCEND’s design and procedure for collecting the speech data, including annotations. ASCEND consists of 10.62 hours of clean speech, collected from 23 bilingual speakers of Chinese and English. Furthermore, we conduct baseline experiments using pre-trained wav2vec 2.0 models, achieving a best performance of 22.69% character error rate and 27.05% mixed error rate. Keywords: code-switching, corpus, bilingual, speech, dialogue, Mandarin Chinese, English, low-resource 1. Introduction Most of our knowledge about speech recognition and speech generation technologies comes from monolingual read speech data collected in a controlled setting (Panay- otov et al., 2015). Monolingual read speech data al- lows researchers to exercise tight control over the linguis- tic backgrounds of speakers and the linguistic material (e.g. reading or repeating sounds, words, or sentences). While being highly informative, these monolingual read speech samples do not capture particular actualities of spo- ken speech (Howell and Kadi-Hanifi, 1991; Blaauw, 1994; Batliner et al., 1995; Li, 2002; Yang and Esposito, 2013; Haynes et al., 2015). Code-switching is a phenomenon typical of spoken speech, characterized by alternating use of more than one lan- guage. It may occur within a single utterance, which is known as intra-sentential code-switching, or between utterance boundaries, which is commonly referred to as inter-sentential code-switching. To address this phe- nomenon, code-switching corpora in many different lan- guages pairs have been introduced, including but not limited to Indonesian-English (Rizal and Stymne, 2020), Filipino-Spanish (Bautista, 2004), Latin-Irish (Horst, 2017), Spanish-English (García et al., 2018), Hindi- English (Si, 2011; Dey and Fung, 2014), and Chinese- English (Lyu et al., 2010). Since code-switching occurs mostly during spontaneous conversational speech, building models using spontaneous speech should be more beneficial than using read speech. While the frequency of code-switching itself in read speech could be manually adjusted by modifying the transcription, spontaneous and read speech still have many other differ- ences characterized by certain factors. For instance, re- duced spectral space and increased spectral variance are observed in Japanese spontaneous speech (Nakamura et al., ⋆ These authors contributed equally. † The work was done when the author was studying in The Hong Kong University of Science and Technology. 2008). Increased spectral variance has also been observed …

Content truncated to 500 words