Lattice-Based Search for Spoken Utterance Retrieval

Lattice-Based Search for Spoken Utterance Retrieval Murat Saraclar AT&T Labs – Research 180 Park Ave. Florham Park, NJ 07932 murat@research.att.com Richard Sproat University of Illinois at Urbana-Champaign Urbana, IL 61801 rws@uiuc.edu Abstract Recent work on spoken document retrieval has suggested that it is adequate to take the single- best output of ASR, and perform text retrieval on this output. This is reasonable enough for the task of retrieving broadcast news stories, where word error rates are relatively low, and the stories are long enough to contain much redundancy. But it is patently not reasonable if one’s task is to retrieve a short snippet of speech in a domain where WER’s can be as high as 50%; such would be the situation with teleconference speech, where one’s task is to find if and when a participant uttered a certain phrase. In this paper we propose an indexing proce- dure for spoken utterance retrieval that works on lattices rather than just single-best text. We demonstrate that this procedure can improve F scores by over five points compared to single- best retrieval on tasks with poor WER and low redundancy. The representation is flexible so that we can represent both word lattices, as well as phone lattices, the latter being impor- tant for improving performance when search- ing for phrases containing OOV words. 1 Introduction Automatic systems for indexing, archiving, searching and browsing of large amounts of spoken communications have become a reality in the last decade. Most such sys- tems use an automatic speech recognition (ASR) compo- nent to convert speech to text which is then used as an input to a standard text based information retrieval (IR) component. This strategy works reasonably well when speech recognition output is mostly correct or the docu- ments are long enough so that some occurrences of the query terms are recognized correctly. Most of the research has concentrated on retrieval of Broadcast News type of spoken documents where speech is relatively clean and the documents are relatively long. In addition it is possible to find large amounts of text with similar content in order to build better language models and enhance retrieval through use of similar documents. We are interested in extending this to telephone con- versations and teleconferences. Our task is locating oc- currences of a query in spoken communications to aid browsing. This is not exactly spoken document retrieval. In fact, it is more similar to word spotting. Each docu- ment is a short segment of audio. Although reasonable retrieval performance can be ob- tained using the best ASR hypothesis for tasks with moderate (∼ 20%) word error rates, tasks with higher (40 − 50%) word error rates require use of multiple ASR hypotheses. Use of ASR lattices makes the system more robust to recognition errors. Almost all ASR systems have a closed vocabulary. This restriction comes from run-time requirements as well as the finite amount of data used for training the language models of the ASR systems. Typically the recognition vocabulary is …

Content truncated to 500 words