Searching the Audio Notebook: Keyword Search in Recorded Conversation

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 947–954, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics SEARCHING THE AUDIO NOTEBOOK: KEYWORD SEARCH IN RECORDED CONVERSATIONS Peng Yu, Kaijiang Chen, Lie Lu, and Frank Seide Microsoft Research Asia, 5F Beijing Sigma Center, 49 Zhichun Rd., 100080 Beijing, P.R.C. {rogeryu,kaijchen,llu,fseide}@microsoft.com Abstract MIT’s Audio Notebook added great value to the note-taking process by retaining audio record- ings, e.g. during lectures or interviews. The key was to provide users ways to quickly and easily access portions of interest in a recording. Sev- eral non-speech-recognition based techniques were employed. In this paper we present a system to search directly the audio record- ings by key phrases. We have identified the user requirements as accurate ranking of phrase matches, domain independence, and reasonable response time. We address these requirements by a hybrid word/phoneme search in lattices, and a supporting indexing scheme. We will in- troduce the ranking criterion, a unified hybrid posterior-lattice representation, and the index- ing algorithm for hybrid lattices. We present results for five different recording sets, includ- ing meetings, telephone conversations, and in- terviews. Our results show an average search accuracy of 84%, which is dramatically better than a direct search in speech recognition tran- scripts (less than 40% search accuracy). 1 Introduction Lisa Stifelman proposed in her thesis the idea of the “Audio Notebook,” where audio recordings of lectures and interviews are retained along with the notes (Stifel- man, 1997). She has shown that the audio recordings are valuable to users if portions of interest can be accessed quickly and easily. Stifelman explored various techniques for this, includ- ing user-activity based techniques (most noteworthy time- stamping notes so they can serve as an index into the recording) and content-based ones (signal processing for accelerated playback, “snap-to-grid” (=phrase boundary) based on prosodic cues). The latter are intended for sit- uations where the former fail, e.g. when the user has no time for taking notes, does not wish to pay attention to it, or cannot keep up with complex subject matter, and as a consequence the audio is left without index. In this pa- per, we investigate technologies for searching the spoken content of the audio recording. Several approaches have been reported in the litera- ture for the problem of indexing spoken words in au- dio recordings. The TREC (Text REtrieval Conference) Spoken-Document Retrieval (SDR) track has fostered re- search on audio-retrieval of broadcast-news clips. Most TREC benchmarking systems use broadcast-news recog- nizers to generate approximate transcripts, and apply text- based information retrieval to these. They achieve re- trieval accuracy similar to using human reference tran- scripts, and ad-hoc retrieval for broadcast news is consid- ered a “solved problem” (Garofolo, 2000). Noteworthy are the rather low word-error rates (20%) in the TREC evaluations, and that recognition errors did not lead to catastrophic failures due to redundancy of news segments and queries. However, in our scenario, requirements are rather dif- ferent. First, word-error rates are …

Content truncated to 500 words