Combining Multiple Knowledge Sources for Dialogue Segmentation in Multimedia Archives

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 1016–1023, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Combining Multiple Knowledge Sources for Dialogue Segmentation in Multimedia Archives Pei-Yun Hsueh School of Informatics University of Edinburgh Edinburgh, UK EH8 9WL p.hsueh@ed.ac.uk Johanna D. Moore School of Informatics University of Edinburgh Edinburgh, UK EH8 9WL J.Moore@ed.ac.uk Abstract Automatic segmentation is important for making multimedia archives comprehensi- ble, and for developing downstream infor- mation retrieval and extraction modules. In this study, we explore approaches that can segment multiparty conversational speech by integrating various knowledge sources (e.g., words, audio and video recordings, speaker intention and context). In particu- lar, we evaluate the performance of a Max- imum Entropy approach, and examine the effectiveness of multimodal features on the task of dialogue segmentation. We also pro- vide a quantitative account of the effect of using ASR transcription as opposed to hu- man transcripts. 1 Introduction Recent advances in multimedia technologies have led to huge archives of audio-video recordings of multiparty conversations in a wide range of areas including clinical use, online video sharing ser- vices, and meeting capture and analysis. While it is straightforward to replay such recordings, find- ing information from the often lengthy archives is a more challenging task. Annotating implicit seman- tics to enhance browsing and searching of recorded conversational speech has therefore posed new chal- lenges to the field of multimedia information re- trieval. One critical problem is how to divide unstructured conversational speech into a number of locally co- herent segments. The problem is important for two reasons: First, empirical analysis has shown that an- notating transcripts with semantic information (e.g., topics) enables users to browse and find information from multimedia archives more efficiently (Baner- jee et al., 2005). Second, because the automatically generated segments make up for the lack of explicit orthographic cues (e.g., story and paragraph breaks) in conversational speech, dialogue segmentation is useful in many spoken language understanding tasks, including anaphora resolution (Grosz and Sid- ner, 1986), information retrieval (e.g., as input for the TREC Spoken Document Retrieval (SDR) task), and summarization (Zechner and Waibel, 2000). This study therefore aims to explore whether a Maximum Entropy (MaxEnt) classifier can inte- grate multiple knowledge sources for segmenting recorded speech. In this paper, we first evaluate the effectiveness of features that have been proposed in previous work, with a focus on features that can be extracted automatically. Second, we examine other knowledge sources that have not been studied sys- tematically in previous work, but which we expect to be good predictors of dialogue segments. In ad- dition, as our ultimate goal is to develop an infor- mation retrieval module that can be operated in a fully automatic fashion, we also investigate the im- pact of automatic speech recognition (ASR) errors on the task of dialogue segmentation. 2 Previous Work In previous work, the problem of automatic dia- logue segmentation is often considered as similar to the problem of topic segmentation. Therefore, re- …

Content truncated to 500 words