Open-Domain Name Error Detection using a Multi-Task RNN

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 737–746, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Open-Domain Name Error Detection using a Multi-Task RNN Hao Cheng Hao Fang Mari Ostendorf Department of Electrical Engineering University of Washington {chenghao,hfang,ostendor}@uw.edu Abstract Out-of-vocabulary name errors in speech recognition create significant problems for downstream language processing, but the fact that they are rare poses challenges for automatic detection, particularly in an open-domain scenario. To address this problem, a multi-task recurrent neural net- work language model for sentence-level name detection is proposed for use in com- bination with out-of-vocabulary word de- tection. The sentence-level model is also effective for leveraging external text data. Experiments show a 26% improvement in name-error detection F-score over a sys- tem using n-gram lexical features. 1 Introduction Most spoken language processing or dialogue sys- tems are based on a finite vocabulary, so occa- sionally a word used will be out of the vocabu- lary (OOV), in which case the automatic speech recognition (ASR) system chooses the best match- ing in-vocabulary sequence of words to cover that region (where acoustic match dominates the deci- sion). The most difficult OOV words to cover are names, since they are less likely to be covered by morpheme-like subword fragments and they often result in anomalous recognition output, e.g. REF: what can we get at Litanfeeth HYP: what can we get it leaks on feet While these errors are rare, they create major prob- lems for language processing, since names tend to be important for many applications. Thus, it is of interest to automatically detect such error regions for additional analysis or human correction. Named entity recognition (NER) systems have been applied to speech output (Palmer and Osten- dorf, 2005; Sudoh et al., 2006), taking advantage of local contextual cues to names (e.g. titles for person names), but as illustrated above, neighbor- ing words are often affected, which obscures lexi- cal cues to name regions. Parada et al. (2011) re- duce this problem somewhat by applying an NER tagger to a word confusion network (WCN) based on a hybrid word/fragment ASR system. In addition to the problem of noisy context, au- tomatic name error detection is challenging be- cause name errors are rare for a good recognizer. To learn the cues to name errors, it is neces- sary to train from the output of the target rec- ognizer, so machine learning is faced with infre- quent positive examples for which training data is very sparse. In addition, in an open domain sys- tem, automatically-learned lexical context features from one domain may be useless in another. In this paper, we address these general problems – detecting rare events in an open-domain task – specifically for name error detection. Prior work addressed the problem of skewed priors by artifi- cially increasing the error rate by holding names out of the vocabulary (Chen et al., 2013) or by factoring the problem into sentence-level name de- tection and OOV word detection …

Content truncated to 500 words