Semantic Similarity for Detecting Recognition Errors in Automatic Speech Transcripts

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 49–56, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Semantic Similarity for Detecting Recognition Errors in Automatic Speech Transcripts Diana Inkpen Alain Désilets School of Information Technology and Engineering Institute for Information Technology University of Ottawa National Research Council of Canada Ottawa, ON, K1N 6H5, Canada Ottawa, ON, K1AOR6, Canada diana@site.uottawa.ca alain.desilets@nrc-cnrc.gc.ca Abstract Browsing through large volumes of spoken audio is known to be a challenging task for end users. One way to alleviate this prob- lem is to allow users to gist a spoken audio document by glancing over a transcript generated through Automatic Speech Rec- ognition. Unfortunately, such transcripts typically contain many recognition errors which are highly distracting and make gist- ing more difficult. In this paper we present an approach that detects recognition errors by identifying words which are semantic outliers with respect to other words in the transcript. We describe several variants of this approach. We investigate a wide range of evaluation measures and we show that we can significantly reduce the number of errors in content words, with the trade-off of losing some good content words. 1 Introduction Spoken audio documents are becoming more and more common place due to the rising popularity of technologies such as: video and audio conferenc- ing, video web-casting and digital cameras for the consumer market. Unfortunately, speech docu- ments are inherently hard to browse because of their transient nature. For example, imagine trying to locate the audio segment in the recording of a 60-minute meeting, where John talked about pro- ject X. Typically, this would require fast forward- ing through the audio by some amount, then listening and trying to remember if the current seg- ment was spoken before or after the desired seg- ment, then fast-forwarding or backtracking by a small amount, and so on. One way to make audio browsing of audio docu- ments more efficient is to allow the user to navi- gate through a textual transcript that is cross- referenced with corresponding time points into the original audio (Nakatani et al. 1998; Hirschberg et al. 1999). Such transcripts can easily be produced with Automatic Speech Recognition (ASR) sys- tems today. Unfortunately, such transcripts typi- cally contain recognition errors that make them hard to browse and understand. Although Word Error Rates (WER) of the order of 20% can be achieved for broadcast quality audio, the WER for more common situations (ex: less-than-broadcast quality recordings of meetings) is typically in the order of 50% or more. The work we present in this paper aims at auto- matically identifying recognition errors and remov- ing them from the transcript, in order to make gisting and browsing of the corresponding audio more efficient. For example, consider the follow- ing portion of a transcript that was produced with the Dragon NaturallySpeaking speech recognition system from the audio of a meeting: “Weenie to decide quickly whether local for large expensive plasma screen aura for a …

Content truncated to 500 words