Combining Speech Retrieval Results with Generalized Additive Models

Proceedings of ACL-08: HLT, pages 461–469, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Combining Speech Retrieval Results with Generalized Additive Models J. Scott Olsson∗ and Douglas W. Oard† UMIACS Laboratory for Computational Linguistics and Information Processing University of Maryland, College Park, MD 20742 Human Language Technology Center of Excellence John Hopkins University, Baltimore, MD 21211 olsson@math.umd.edu, oard@umd.edu Abstract Rapid and inexpensive techniques for auto- matic transcription of speech have the po- tential to dramatically expand the types of content to which information retrieval tech- niques can be productively applied, but lim- itations in accuracy and robustness must be overcome before that promise can be fully realized. Combining retrieval results from systems built on various errorful representa- tions of the same collection offers some po- tential to address these challenges. This pa- per explores that potential by applying Gener- alized Additive Models to optimize the combi- nation of ranked retrieval results obtained us- ing transcripts produced automatically for the same spoken content by substantially differ- ent recognition systems. Topic-averaged re- trieval effectiveness better than any previously reported for the same collection was obtained, and even larger gains are apparent when using an alternative measure emphasizing results on the most difficult topics. 1 Introduction Speech retrieval, like other tasks that require trans- forming the representation of language, suffers from both random and systematic errors that are intro- duced by the speech-to-text transducer. Limita- tions in signal processing, acoustic modeling, pro- nunciation, vocabulary, and language modeling can be accommodated in several ways, each of which make different trade-offs and thus induce different ∗ Dept. of Mathematics/AMSC, UMD † College of Information Studies, UMD error characteristics. Moreover, different applica- tions produce different types of challenges and dif- ferent opportunities. As a result, optimizing a sin- gle recognition system for all transcription tasks is well beyond the reach of present technology, and even systems that are apparently similar on average can make different mistakes on different sources. A natural response to this challenge is to combine re- trieval results from multiple systems, each imper- fect, to achieve reasonably robust behavior over a broader range of tasks. In this paper, we compare alternative ways of combining these ranked lists. Note, we do not assume access to the internal work- ings of the recognition systems, or even to the tran- scripts produced by those systems. System combination has a long history in infor- mation retrieval. Most often, the goal is to combine results from systems that search different content (“collection fusion”) or to combine results from dif- ferent systems on the same content (“data fusion”). When working with multiple transcriptions of the same content, we are again presented with new op- portunities. In this paper we compare some well known techniques for combination of retrieval re- sults with a new evidence combination technique based on a general framework known as Gener- alized Additive Models (GAMs). We show that this new technique significantly outperforms sev- eral well known information retrieval fusion tech- niques, and we …

Content truncated to 500 words