Automatic Estimation of Word Significance oriented for Speech-based Information Retrieval

Automatic Estimation of Word Significance oriented for Speech-based Information Retrieval Takashi Shichiri Graduate School of Science and Tech. Ryukoku University Seta, Otsu 520-2194, Japan shichiri@nlp.i.ryukoku.ac.jp Hiroaki Nanjo Faculty of Science and Tech. Ryukoku University Seta, Otsu 520-2194, Japan nanjo@nlp.i.ryukoku.ac.jp Takehiko Yoshimi Faculty of Science and Tech. Ryukoku University Seta, Otsu 520-2194, Japan yoshimi@nlp.i.ryukoku.ac.jp Abstract Automatic estimation of word significance oriented for speech-based Information Re- trieval (IR) is addressed. Since the sig- nificance of words differs in IR, automatic speech recognition (ASR) performance has been evaluated based on weighted word er- ror rate (WWER), which gives a weight on errors from the viewpoint of IR, instead of word error rate (WER), which treats all words uniformly. A decoding strategy that minimizes WWER based on a Minimum Bayes-Risk framework has been shown, and the reduction of errors on both ASR and IR has been reported. In this paper, we propose an automatic estimation method for word significance (weights) based on its influence on IR. Specifically, weights are estimated so that evaluation measures of ASR and IR are equivalent. We apply the proposed method to a speech-based information retrieval sys- tem, which is a typical IR system, and show that the method works well. 1 Introduction Based on the progress of spoken language process- ing, the main target of speech processing has shifted from speech recognition to speech understanding. Since speech-based information retrieval (IR) must extract user intention from speech queries, it is thus a typical speech understanding task. IR typically searches for appropriate documents such as news- paper articles or Web pages using statistical match- ing for a given query. To define the similarity be- tween a query and documents, the word vector space model or “bag-of-words” model is widely adopted, and such statistics as the TF-IDF measure are intro- duced to consider the significance of words in the matching. Therefore, when using automatic speech recognition (ASR) as a front-end of such IR systems, the significance of the words should be considered in ASR; words that greatly affect IR performance must be detected with higher priority. Based on such a background, ASR evaluation should be done from the viewpoint of the quality of mis-recognized words instead of quantity. From this point of view, word error rate (WER), which is the most widely used evaluation measure of ASR accuracy, is not an appropriate evaluation measure when we want to use ASR systems for IR because all words are treated identically in WER. Instead of WER, weighted WER (WWER), which consid- ers the significance of words from a viewpoint of IR, has been proposed as an evaluation measure for ASR. Nanjo et.al showed that the ASR based on the Minimum Bayes-Risk framework could reduce WWER and the WWER reduction was effective for key-sentence indexing and IR (H.Nanjo et al., 2005). To exploit ASR which minimizes WWER for IR, we should appropriately define weights of words. Ideal weights would give a WWER equivalent to IR performance degradation when a corresponding ASR result is used as …

Content truncated to 500 words