Integrating Multi-level Linguistic Knowledge with a Unified Framework for Mandarin Speech Recognition

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 820–828, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Integrating Multi-level Linguistic Knowledge with a Unified Framework for Mandarin Speech Recognition Xinhao Wang, Jiazhong Nie, Dingsheng Luo, Xihong Wu∗ Speech and Hearing Research Center, Key Laboratory of Machine Perception (Ministry of Education), School of Electronics Engineering and Computer Science, Peking University, Beijing, 100871, China {wangxh,niejz,wxh,dsluo}@cis.pku.edu.cn Abstract To improve the Mandarin large vocabulary continuous speech recognition (LVCSR), a unified framework based approach is intro- duced to exploit multi-level linguistic knowl- edge. In this framework, each knowledge source is represented by a Weighted Finite State Transducer (WFST), and then they are combined to obtain a so-called analyzer for in- tegrating multi-level knowledge sources. Due to the uniform transducer representation, any knowledge source can be easily integrated into the analyzer, as long as it can be encoded into WFSTs. Moreover, as the knowledge in each level is modeled independently and the combination is processed in the model level, the information inherently in each knowledge source has a chance to be thoroughly ex- ploited. By simulations, the effectiveness of the analyzer is investigated, and then a LVCSR system embedding the presented ana- lyzer is evaluated. Experimental results reveal that this unified framework is an effective ap- proach which significantly improves the per- formance of speech recognition with a 9.9% relative reduction of character error rate on the HUB-4 test set, a widely used Mandarin speech recognition task. 1 Introduction Language modeling is essential for large vocabu- lary continuous speech recognition (LVCSR), which aims to determine the prior probability of a supposed word string W, p(W). Although the word-based n- gram language model remains the mainstream for ∗Corresponding author: Xihong Wu most speech recognition systems, the utilization of linguistic knowledge is too limited in this model. Consequently, many researchers have focused on introducing more linguistic knowledge in language modeling, such as lexical knowledge , syntax and semantics of language (Wang and Vergyri, 2006; Wang et al., 2004; Charniak, 2001; Roark, 2001; Chelba, 2000; Heeman, 1998; Chelba et al., 1997). Recently, structured language models have been introduced to make use of syntactic hierarchi- cal characteristics (Roark, 2001; Charniak, 2001; Chelba, 2000). Nevertheless, the computational complexity of decoding will be heavily increased, as they are parser-based models. In contrast, the class- based language model groups the words that have similar functions of syntax or semantics into mean- ingful classes. As a result, it handles the questions of data sparsity and generalization of unseen event. In practice, the part-of-speech (POS) information, cap- turing the syntactic role of words, has been widely used in clustering words (Wang and Vergyri, 2006; Maltese et al., 2001; Samuelsson and Reichl, 1999). In Heeman’s POS language model (Heeman, 1998), the joint probability of word sequence and associ- ated POS sequence was estimated directly, which has been demonstrated to be superior to the condi- tional probability previously used in the class-based models (Johnson, 2001). Moreover, a SuperARV language model was presented …

Content truncated to 500 words