Error Detection Using Linguistic Features

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 41–48, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Error Detection Using Linguistic Features Yongmei Shi Department of Computer Science and Electrical Engineering University of Maryland Baltimore County Baltimore, MD 21250 yshi1@umbc.edu Lina Zhou Information Systems Department University of Maryland Baltimore County Baltimore, MD 21250 zhoul@umbc.edu Abstract Recognition errors hinder the prolifera- tion of speech recognition (SR) systems. Based on the observation that recogni- tion errors may result in ungrammatical sentences, especially in dictation appli- cation where an acceptable level of ac- curacy of generated documents is indis- pensable, we propose to incorporate two kinds of linguistic features into error de- tection: lexical features of words, and syn- tactic features from a robust lexicalized parser. Transformation-based learning is chosen to predict recognition errors by in- tegrating word confidence scores with lin- guistic features. The experimental results on a dictation data corpus show that lin- guistic features alone are not as useful as word confidence scores in detecting er- rors. However, linguistic features provide complementary information when com- bined with word confidence scores, which collectively reduce the classification error rate by 12.30% and improve the F measure by 53.62%. 1 Introduction The proliferation of speech recognition (SR) sys- tems is hampered by the ever-presence of recogni- tion errors and the significant amount of effort in- volved in error correction. A user study (Sears et al., 2001) showed that users spent one-third of their time finding and locating errors and another one-third of the time correcting errors in a hand-free dictation task. Successfully detecting SR errors can speed up the entire process of error correction. Therefore, we focus on error detection in this study. A common approach to detecting SR errors is an- notating confidence at the word level. The major- ity of confidence annotation methods are based on feature combination, which follows two steps: (i) extract useful features characteristics of the correct- ness of words either from the inner components of an SR system (SR-dependent features) or from the recognition output (SR-independent features); and (ii) develop a binary classifier to separate words into two groups: correct recognitions and errors. Various features extracted from different compo- nents of an SR system, such as the acoustic model, the language model, and the decoder, have been proven useful to detecting recognition errors (Chase, 1997; Pao et al., 1998; San-Segundo et al., 2001). Nonetheless, merely using these features is inade- quate, because the information conveyed by these features has already been considered when SR sys- tems generate the output. A common observation is that the combination of SR-dependent features can only marginally improve the performance achieved by using only the best single feature (Zhang and Rudnicky, 2001; Sarikaya et al., 2003). Hence in- formation sources beyond the SR system are desired in error detection. High-level linguistic knowledge is a good candi- date for additional information sources. It can be extracted from the SR output via natural language processing, …

Content truncated to 500 words