Making Sense of Sound: Unsupervised Topic Segmentation over Acoustic Input

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 504–511, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Making Sense of Sound: Unsupervised Topic Segmentation over Acoustic Input Igor Malioutov, Alex Park, Regina Barzilay, and James Glass Massachusetts Institute of Technology {igorm,malex,regina,glass}@csail.mit.edu Abstract We address the task of unsupervised topic segmentation of speech data operating over raw acoustic information. In contrast to ex- isting algorithms for topic segmentation of speech, our approach does not require in- put transcripts. Our method predicts topic changes by analyzing the distribution of re- occurring acoustic patterns in the speech sig- nal corresponding to a single speaker. The algorithm robustly handles noise inherent in acoustic matching by intelligently aggregat- ing information about the similarity profile from multiple local comparisons. Our ex- periments show that audio-based segmen- tation compares favorably with transcript- based segmentation computed over noisy transcripts. These results demonstrate the desirability of our method for applications where a speech recognizer is not available, or its output has a high word error rate. 1 Introduction An important practical application of topic segmen- tation is the analysis of spoken data. Paragraph breaks, section markers and other structural cues common in written documents are entirely missing in spoken data. Insertion of these structural markers can benefit multiple speech processing applications, including audio browsing, retrieval, and summariza- tion. Not surprisingly, a variety of methods for topic segmentation have been developed in the past (Beeferman et al., 1999; Galley et al., 2003; Dielmann and Renals, 2005). These methods typi- cally assume that a segmentation algorithm has ac- cess not only to acoustic input, but also to its tran- script. This assumption is natural for applications where the transcript has to be computed as part of the system output, or it is readily available from other system components. However, for some domains and languages, the transcripts may not be available, or the recognition performance may not be adequate to achieve reliable segmentation. In order to process such data, we need a method for topic segmentation that does not require transcribed input. In this paper, we explore a method for topic seg- mentation that operates directly on a raw acoustic speech signal, without using any input transcripts. This method predicts topic changes by analyzing the distribution of reoccurring acoustic patterns in the speech signal corresponding to a single speaker. In the same way that unsupervised segmentation algo- rithms predict boundaries based on changes in lexi- cal distribution, our algorithm is driven by changes in the distribution of acoustic patterns. The central hypothesis here is that similar sounding acoustic se- quences produced by the same speaker correspond to similar lexicographic sequences. Thus, by ana- lyzing the distribution of acoustic patterns we could approximate a traditional content analysis based on the lexical distribution of words in a transcript. Analyzing high-level content structure based on low-level acoustic features poses interesting compu- tational and linguistic challenges. For instance, we need to handle the noise inherent in matching based on …

Content truncated to 500 words