Improving Information Extraction by Modeling Errors in Speech Recognizer Output

Improving Information Extraction by Modeling Errors in Speech Recognizer Output David D. Palmer �� � The MITRE Corporation 202 Burlington Road Bedford, MA 01730 palmer@mitre.org Mari Ostendorf � � Electrical Engineering Dept. University of Washington Seattle, WA 98195 mo@ee.washington.edu ABSTRACT In this paper we describe a technique for improving the performance of an information extraction system for speech data by explicitly modeling the errors in the recognizer output. The approach combines a statistical model of named entity states with a lattice representation of hypothesized words and errors annotated with recognition confidence scores. Additional refinements include the use of multiple error types, improved confidence estimation, and multi- pass processing. In combination, these techniques im- prove named entity recognition performance over a text- based baseline by 28%. Keywords ASR error modeling, information extraction, word confi- dence 1. INTRODUCTION There has been a great deal of research on applying nat- ural language processing (NLP) techniques to text-based sources of written language data, such as newspaper and newswire data. Most NLP approaches to spoken language data, such as broadcast news and telephone conversations, have consisted of applying text-based systems to the out- put of an automatic speech recognition (ASR) system; re- search on improving these approaches has focused on ei- ther improving the ASR accuracy or improving the text- based system (or both). However, applying text-based sys- tems to ASR output ignores the fact that there are funda- mental differences between written texts and ASR tran- scriptions of spoken language: the style is different be- tween written and spoken language, the transcription con- ventions are different, and, most importantly, there are er- rors in ASR transcriptions. In this work, we focus on the third problem: handling errors by explicitly modeling un- certainty in ASR transcriptions. . The idea of explicit error handling in information ex- traction (IE) from spoken documents was introduced by Grishman in [1], where a channel model of word insertions and deletions was used with a deterministic pattern match- ing system for information extraction. While the use of an error model resulted in substantial performance improve- ments, the overall performance was still quite low, perhaps because the original system was designed to take advan- tage of orthographic features. In looking ahead, Grishman suggests that a probabilistic approach might be more suc- cessful at handling errors. The work described here provides such an approach, but introduces an acoustically-driven word confidence score rather than the word-based channel model proposed in [1]. More specifically, we provide a unified approach to pre- dicting and using uncertainty in processing spoken lan- guage data, focusing on the specific IE task of identifying named entities (NEs). We show that by explicitly mod- eling multiple types of errors in the ASR output, we can improve the performance of an IE system, which benefits further from improved error prediction using new features derived from multi-pass processing. The rest of the paper is organized as follows. In Sec- tion 2 we describe our error modeling, including explicit modeling of multiple ASR …

Content truncated to 500 words