Combining Multiple Models for Speech Information Retrieval

Combining Multiple Models for Speech Information Retrieval Muath Alzghool and Diana Inkpen School of Information Technology and Engineering University of Ottawa {alzghool,diana}@ site.uottawa.ca Abstract In this article we present a method for combining different information retrieval models in order to increase the retrieval performance in a Speech Information Retrieval task. The formulas for combining the models are tuned on training data. Then the system is evaluated on test data. The task is particularly difficult because the text collection is automatically transcribed spontaneous speech, with many recognition errors. Also, the topics are real information needs, difficult to satisfy. Information Retrieval systems are not able to obtain good results on this data set, except for the case when manual summaries are included. 1. Introduction Conversational speech such as recordings of interviews or teleconferences is difficult to search through. The transcripts produced with Automatic Speech Recognition (ASR) systems tend to contain many recognition errors, leading to low Information Retrieval (IR) performance (Oard et al., 2007). Previous research has explored the idea of combining the results of different retrieval strategies; the motivation is that each technique will retrieve different sets of relevant documents; therefore combining the results could produce a better result than any of the individual techniques. We propose new data fusion techniques for combining the results of different IR models. We applied our data fusion techniques to the Mallach collection (Oard et al., 2007) used in the Cross-Language Speech Retrieval (CLSR) task at Cross-Language Evaluation Forum (CLEF) 2007. The Mallach collection comprises 8104 “documents” which are manually-determined topically-coherent segments taken from 272 interviews with Holocaust survivors, witnesses and rescuers, totalling 589 hours of speech. Figure 1 shows the document structure in CLSR test collection, two ASR transcripts are available for this data, in this work we use the ASRTEXT2004A field provided by IBM research with a word error rate of 38%. Additionally, metadata fields for each document include: two sets of 20 automatically assigned keywords determined using two different kNN classifiers (AK1 and AK2), a set of a varying number of manually-assigned keywords (MK), and a manual 3-sentence summary written by an expert in the field. A set of 63 training topics and 33 test topics were generated for this task. The topics provided with the collection were created in English from actual user requests. Topics were structured using the standard TREC format of Title, Description and Narrative fields. To enable CL-SR experiments the topics were translated into Czech, German, French, and Spanish by native speakers; Figure 2 and 3 show two examples for English and its translation in French respectively. Relevance judgments were generated using a search-guided procedure and standard pooling methods. See (Oard et al., 2004) for full details of the collection design. We present results on the automatic transcripts for English queries and translated queries (cross-language) for two combination methods; we also present results when manual summaries and manual keywords are indexed. <DOC> <DOCNO>VHF[IntCode]-[SegId].[SequenceNum]</DOCNO\> <INTERVIEWDATA>Interviewee name(s) and birthdate</INTERVIEWDATA> <NAME>Full name of every person mentioned</NAME> <MANUALKEYWORD>Thesaurus keywords assigned to the …

Content truncated to 500 words