A Cross-language Study on Automatic Speech Disfluency Detection

Proceedings of NAACL-HLT 2013, pages 703–708, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics A Cross-language Study on Automatic Speech Disfluency Detection Wen Wang SRI International Menlo Park, CA wwang@speech.sri.com Andreas Stolcke Microsoft Research Mountain View, CA anstolck@microsoft.com Jiahong Yuan, Mark Liberman University of Pennsylvania Philadelphia, PA jiahong.yuan@gmail.com markyliberman@gmail.com Abstract We investigate two systems for automatic dis- fluency detection on English and Mandarin conversational speech data. The first system combines various lexical and prosodic fea- tures in a Conditional Random Field model for detecting edit disfluencies. The second system combines acoustic and language model scores for detecting filled pauses through constrained speech recognition. We compare the contri- butions of different knowledge sources to de- tection performance between these two lan- guages. 1 Introduction Speech disfluencies are common phenomena in spontaneous speech. They consist of spoken words and phrases that represent self-correction, hesitation, and floor-grabbing behaviors, but do not add seman- tic information; removing them yields the intended, fluent utterance. The presence of disfluencies in conversational speech data can cause problems for both downstream processing (parsing and other nat- ural language processing tasks) and human readabil- ity of speech transcripts. There has been much re- search effort on automatic disfluency detection in recent years (Shriberg and Stolcke, 1997; Snover et al., 2004; Liu et al., 2006; Lin and Lee, 2009; Schuler et al., 2010; Georgila et al., 2010; Zwarts and Johnson, 2011), particularly from the DARPA EARS (Effective, Affordable, Reusable Speech-to- Text) MDE (MetaData Extraction) (DARPA Infor- mation Processing Technology Office, 2003) pro- gram, which focused on the automatic transcription of sizable amounts of speech data and rendering such transcripts in readable form, for both conversa- tional telephone speech (CTS) and broadcast news (BN). However, the EARS MDE effort was focused on English only, and there hasn’t been much research on the effectiveness of similar automatic disfluency detection approaches for multiple languages. This paper presents three main innovations. First, we extend the EARS MDE-style disfluency detection approach combining lexical and prosodic features using a Conditional Random Field (CRF) model, which was employed for detecting disfluency on En- glish conversational speech data (Liu et al., 2005), to Mandarin conversational speech, as presented in Section 2. Second, we implement an automatic filled pause detection approach through constrained speech recognition, as presented in Section 3. Third, for both disfluency detection systems, we compare side-by-side contributions of different knowledge sources to detection performance for two languages, English and Mandarin, as presented in Section 4. Conclusions appear in Section 5. 2 EARS MDE Style Automatic Disfluency Detection We focus on two types of disfluencies, Fillers and Edit disfluencies, following the EARS MDE disflu- ency types modeled in (Liu et al., 2006). Fillers in- clude filled pauses (FP), discourse markers (DM), and explicit editing terms (ET). FPs are words used by the speakers as floor holders to maintain con- trol of a conversation. They can also indicate hes- itations of the speaker. In this work, English FPs 703 comprise uh and um, based on English …

Content truncated to 500 words