Formatting Time-Aligned ASR Transcripts for Readability

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 198–206, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Formatting Time-Aligned ASR Transcripts for Readability Maria Shugrina∗ Google Inc. New York, NY 10011 shumash@google.com Abstract We address the problem of formatting the out- put of an automatic speech recognition (ASR) system for readability, while preserving word- level timing information of the transcript. Our system enriches the ASR transcript with punc- tuation, capitalization and properly written dates, times and other numeric entities, and our approach can be applied to other format- ting tasks. The method we describe combines hand-crafted grammars with a class-based lan- guage model trained on written text and relies on Weighted Finite State Transducers (WF- STs) for the preservation of start and end time of each word. 1 Introduction and Prior Work The output of a typical ASR system lacks punctua- tion, capitalization and proper formatting of entities such as phone numbers, time expressions and dates. Even if such automatic transcript is free of recogni- tion errors, it is difficult for a human to parse. The proper formatting of the transcript gains particular importance in applications where the user relies on ASR output for information and where information- rich numeric entities (e.g. time expressions, mone- tary amounts) are common. A good example of such application is a voicemail transcription system. The goal of our work is to transform the raw transcript into its proper written form in order to optimize it for the visual scanning task by the end user. We present quantitative and qualitative evaluation of our system with a focus on numeric entity formatting, punctua- tion and capitalization (See Fig. 1). Apart from text, the ASR output usually con- tains word-level metadata such as time-alignment and confidence. Such quantities may be useful for a variety of applications. Although simple to recover ∗Thank you to Michiel Bacchiani, Martin Jansche, Michael Riley and Cyril Allauzen for discussion and support. Raw Transcript: hi bill it’s tracy at around three thirty P M just got an apartment for one thousand three thirty one thou- sand four hundred a month my number is five five five eight eight eight eight extension is three thirty bye Our Result: Hi Bill, it’s Tracy at around 3:30 PM, just got an apartment for 1,330 1,400 a month. My number is 555-8888 extension is 330. Bye. Figure 1: An example of a raw transcript with ambiguous written forms and the output of our formatting system. via word alignment after some types of formatting, word-level quantities may be difficult to preserve if the original text has undergone a significant transfor- mation. We present a formal and general augmen- tation of our WFST-based technique that preserves word-level timing and confidence information dur- ing arbitrary formatting. The problems of sentence boundary detection and punctuation of transcripts have received a substantial amount of attention, e.g. (Beeferman et al., 1998; Shriberg et al., 2000; Christensen et al., 2001; Liu et al., 2006; …

Content truncated to 500 words