Developing Speech Recognition and Synthesis Technologies to Support Computer-Aided Pronunciation Training for Chinese Learners of English

Developing Speech Recognition and Synthesis Technologies to Support Computer-Aided Pronunciation Training for Chinese Learners of English* Helen Meng Human-Computer Communications Laboratory The Chinese University of Hong Kong hmmeng@se.cuhk.edu.hk * Joint Work with Alissa Harrison, Pauline Lee, Wai-Kit Lo, Lan Wang and Virginia Yip Copyright 2009 by Helen Meng Abstract. We describe ongoing research in the development of speech technologies that strives to raise the efficacy of computer-aided pronunciation training, especially for Chinese learners of English. Our approach is grounded on the theory of language transfer and involves a systematic phonological comparison between the primary language (L1 being Chinese) and secondary language (L2 being English) to predict possible segmental and suprasegmental realizations that constitute mispronunciations in L2 English. The predictions are validated based on a specially designed corpus that consists of several hundred hours of L2 English speech. The speech data supports the development of automatic speech recognition technologies that can detect and diagnose mispronunciations. The diagnosis aims to support the design of pedagogical and remedial instructions, which involves text-to- speech synthesis technologies in audiovisual forms. 1 Introduction English is the lingua franca of our world. It is the second language (L2) most actively studied across Asia, as well as an official language or working language used in education, government, media, etc. in many regions. Hence, acquiring communicative competence in English is of prime importance. It has been estimated that by 2010 there will be 2 billion English learners worldwide, and the proportion in Asia alone will exceed the number of native speakers (Asia Economic News, 2006). Second language learning, specifically pronunciation learning, involves correct perception and production of sounds in the target language. The learning process tends to be influenced by well established perceptions of sounds and articulatory motions in the primary language (L1). This cross-linguistic influence is often referred as language transfer. Negative transfer of L1 features causes inaccuracies and errors in L2 speech productions, which impede intelligibility. Consequently, the study of L2 speech productions (i.e. the “interlanguage” of learners who have not acquired native-like proficiency) is of great interest to phoneticians, linguists, language educators, as well as technologists engaged in the development of CAPT (computer-aided pronunciation training) systems. These applications can complement classroom teaching and provide unique benefits to the learner in terms of accessibility, reduced anxiety and individualized instructions. Effective pronunciation training tools need to provide learners with detailed mispronunciation detection, diagnosis and corrective feedback (Ehsani and Knodt, 1998). Previous work has shown that automatic pronunciation scores at the word-level or sentence- level correlate highly with human raters but fail to lead to measurable improvement in learner’s overall pronunciation (Precoda et al., 2000). However, locating mispronunciations at the phone-level to learners has been shown to lead to statistically significant improvement for the 40 23rd Pacific Asia Conference on Language, Information and Computation, pages 40–42 production of those targeted phones (Neri et al., 2006). Moreover, diagnostic feedback to learners (e.g. “you inserted a vowel at the end of the word”) has also been shown to lead to significant …

Content truncated to 500 words