Non-Native Users in the Let’s Go!! Spoken Dialogue System: Dealing with Linguistic Mismatch

Non-Native Users in the Let’s Go!! Spoken Dialogue System: Dealing with Linguistic Mismatch Antoine Raux and Maxine Eskenazi Language Technologies Institute Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA 15232, USA {antoine+,max+}@cs.cmu.edu Abstract This paper describes the CMU Let’s Go!! bus information system, an experimental system designed to study the use of spoken dialogue interfaces by non-native speakers. The differ- ences in performance of the speech recogni- tion and language understanding modules of the system when confronted with native and non-native spontaneous speech are analyzed. Focus is placed on the linguistic mismatch be- tween the user input and the system’s expecta- tions, and on its implications in terms of lan- guage modeling and parsing performance. The effect of including non-native data when build- ing the speech recognition and language under- standing modules is discussed. In order to close the gap between non-native and native input, a method is proposed to automatically generate confirmation prompts that are both close to the user’s input and covered by the system’s lan- guage model and grammar, in order to help the user acquire idiomatic expressions appropriate to the task. 1 Introduction 1.1 Spoken Dialogue Systems and Non-Native Speakers Spoken dialogue systems rely on models of human lan- guage to understand users’ spoken input. Such models cover the acoustic and linguistic space of the common language used by the system and the user. In current systems, these models are learned from large corpora of recorded and transcribed conversations matching the do- main of the system. In most of the cases, these cor- pora are gathered from native speakers of the language because they are the main target of the system and be- cause developers and researchers are often native speak- ers themselves. However, when the common language is not the users’ native language, their utterances might fall out of this “standard” native model, seriously degrad- ing the recognition accuracy and overall system perfor- mance. As telephone-based information access systems become more common and available to the general pub- lic, this inability to deal with non-native speakers (or with any “non-standard” subgroup such as the elderly) is a serious limitation since, at least for some applications, (e.g. tourist information, legal/social advice) non-native speakers represent a significant portion of the everyday user population. 1.2 Previous Work on Non-Native Speech Recognition Over the past ten years, extensive work has been done on non-native speech recognition. Early research aimed at endowing Computer Assisted Language Learning soft- ware with speech recognition capabilities (e.g. (Eske- nazi and Hansma, 1998), (Witt and Young, 1997)). Usu- ally such systems are targeted at one specific popula- tion, that is, people who share the same native language (L1). Thus, most research in non-native speech recog- nition uses knowledge of the L1, as well as databases of accented speech specially recorded from speakers of the target population. Ideally, by training acoustic mod- els on target non-native speech, one would capture its specific characteristics just as training on native speech does. However collecting amounts of non-native speech …

Content truncated to 500 words