Language Modeling with Functional Head Constraint for Code Switching Speech Recognition

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 907–916, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Language Modeling with Functional Head Constraint for Code Switching Speech Recognition Ying Li and Pascale Fung Human Language Technology Center Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology eewing@ee.ust.hk, pascale@ece.ust.hk Abstract In this paper, we propose novel struc- tured language modeling methods for code mixing speech recognition by incorporat- ing a well-known syntactic constraint for switching code, namely the Functional Head Constraint (FHC). Code mixing data is not abundantly available for training language models. Our proposed meth- ods successfully alleviate this core prob- lem for code mixing speech recognition by using bilingual data to train a struc- tured language model with syntactic con- straint. Linguists and bilingual speakers found that code switch do not happen be- tween the functional head and its comple- ments. We propose to learn the code mix- ing language model from bilingual data with this constraint in a weighted finite state transducer (WFST) framework. The constrained code switch language model is obtained by first expanding the search net- work with a translation model, and then using parsing to restrict paths to those per- missible under the constraint. We im- plement and compare two approaches - lattice parsing enables a sequential cou- pling whereas partial parsing enables a tight coupling between parsing and fil- tering. We tested our system on a lec- ture speech dataset with 16% embedded second language, and on a lunch conver- sation dataset with 20% embedded lan- guage. Our language models with lattice parsing and partial parsing reduce word error rates from a baseline mixed lan- guage model by 3.8% and 3.9% in terms of word error rate relatively on the aver- age on the first and second tasks respec- tively. It outperforms the interpolated lan- guage model by 3.7% and 5.6% in terms of word error rate relatively, and outperforms the adapted language model by 2.6% and 4.6% relatively. Our proposed approach avoids making early decisions on code- switch boundaries and is therefore more robust. We address the code switch data scarcity challenge by using bilingual data with syntactic structure. 1 Introduction In multilingual communities, it is common for people to mix two or more languages in their speech. A single sentence spoken by bilingual speakers often contains the main, matrix language and an embedded second language. This type of linguistic phenomenon is called ”code switch- ing” by linguists. It is increasingly important for automatic speech recognition (ASR) systems to recognize code switching speech as they exist in scenarios such as meeting and interview speech, lecture speech, and conversational speech. Code switching is common among bilingual speakers of Spanish-English, Hindi-English, Chinese-English, and Arabic-English, among others. In China, lectures, meetings and conversations with techni- cal contents are frequently peppered with English terms even though the general population is not considered bilingual in Chinese and English. Un- like the thousands and tens of …

Content truncated to 500 words