Modeling Cantonese Pronunciation Variations for Large-Vocabulary Continuous Speech Recognition

Computational Linguistics and Chinese Language Processing Vol. 11, No. 1, March 2006, pp. 17-36 17 © The Association for Computational Linguistics and Chinese Language Processing [Received June 16, 2005; Revised January 20, 2006; Accepted January 23, 2006] Modeling Cantonese Pronunciation Variations for Large-Vocabulary Continuous Speech Recognition Tan Lee∗, Patgi Kam∗ and Frank K. Soong∗+ Abstract This paper presents different methods of handling pronunciation variations in Cantonese large-vocabulary continuous speech recognition. In an LVCSR system, three knowledge sources are involved: a pronunciation lexicon, acoustic models and language models. In addition, a decoding algorithm is used to search for the most likely word sequence. Pronunciation variation can be handled by explicitly modifying the knowledge sources or improving the decoding method. Two types of pronunciation variations are defined, namely, phone changes and sound changes. Phone change means that one phoneme is realized as another phoneme. A sound change happens when the acoustic realization is ambiguous between two phonemes. Phone changes are handled by constructing a pronunciation variation dictionary to include alternative pronunciations at the lexical level or dynamically expanding the search space to include those pronunciation variants. Sound changes are handled by adjusting the acoustic models through sharing or adaptation of the Gaussian mixture components. Experimental results show that the use of a pronunciation variation dictionary and the method of dynamic search space expansion can improve speech recognition performance substantially. The methods of acoustic model refinement were found to be relatively less effective in our experiments. Keywords: Automatic Speech Recognition, Pronunciation Variation, Cantonese 1. Introduction Given a speech input, automatic speech recognition (ASR) is a process of generating possible hypotheses for the underlying word sequence. This can be done by establishing a mapping between the acoustic features and the yet to be determined linguistic representations. Given ∗ Department of Electronic Engineering, The Chinese University of Hong Kong, Shatin, N. T., Hong Kong Tel: 852-26098267 Fax: 852-26035558 E-mail: tanlee@ee.cuhk.edu.hk The author for correspondence is Tan Lee. + Micrpsoft Research Asia, 5th Floor, Sigma Center, 49 Zhichun Road, Haidian, Beijing 100080, China 18 Tan Lee et al. the high variability of human speech, such mapping is in general not one-to-one. Different linguistic symbols can give rise to similar speech sounds, while the same linguistic symbol may also be realized in different pronunciations. The variability is due to co-articulation, regional accents, speaking rate, speaking style, etc. Pronunciation modeling is aimed at providing an effective mechanism by which ASR systems can be adapted to pronunciation variability. Pronunciation variations can be divided into two types: phone change and sound change [Kam 2003] [Liu and Fung 2003]. In [Saraçlar and Khudanpur 2000] [Liu 2002], they are also referred to as complete change and partial change, respectively. A phone change happens when a baseform (canonical) phoneme is realized as another phoneme, which is referred to as its surface-form. The baseform pronunciation is considered to be the “standard” pronunciation that the speaker is supposed to use. Surface-form pronunciations are the actual pronunciations that different speakers may use. A sound change can …

Content truncated to 500 words