Prosody-Based Unsupervised Speech Summarization with Two-Layer Mutually Reinforced Random Walk

International Joint Conference on Natural Language Processing, pages 648–654, Nagoya, Japan, 14-18 October 2013. Prosody-Based Unsupervised Speech Summarization with Two-Layer Mutually Reinforced Random Walk Sujay Kumar Jauhar, Yun-Nung Chen, and Florian Metze School of Computer Science, Carnegie Mellon University 5000 Forbes Ave., Pittsburgh, PA 15213-3891, USA {sjauhar,yvchen,fmetze}@cs.cmu.edu Abstract This paper presents a graph-based model that integrates prosodic features into an unsupervised speech summarization framework without any lexical informa- tion. In particular it builds on previous work using mutually reinforced random walks, in which a two-layer graph struc- ture is used to select the most salient ut- terances of a conversation. The model consists of one layer of utterance nodes and another layer of prosody nodes. The random walk algorithm propagates scores between layers to use shared information for selecting utterance nodes with highest scores as summaries. A comparative eval- uation of our prosody-based model against several baselines on a corpus of academic multi-party meetings reveals that it per- forms competitively on very short sum- maries, and better on longer summaries according to ROUGE scores as well as the average relevance of selected utterances. 1 Introduction Automatic extractive speech summarization (Hori and Furui, 2001) has garnered considerable in- terest in the natural language processing research community for its immediate application in mak- ing large volumes of multimedia documents more accessible. Several variants of speech summa- rization have been studied in a range of tar- get domains, including news (Hori et al., 2002; Maskey and Hirschberg, 2003), lectures (Glass et al., 2007; Chen et al., 2011) and multi-party meet- ings (Banerjee and Rudnicky, 2008; Liu and Liu, 2010; Chen and Metze, 2012b). Research in speech summarization – unlike its text-based counterpart – carries intrinsic difficul- ties, which draw their origins from the noisy na- ture of the data under consideration: imperfect ASR transcripts due to recognition errors, lack of proper segmentation, etc. However, it also of- fers some advantages by making it possible to leverage extra-textual information such as emo- tion and other speaker states through an incorpo- ration of prosodic knowledge into the summariza- tion model. A study by Maskey and Hirschberg (2005) on the relevance of various levels of linguistic knowl- edge (including lexical, prosodic and discourse structure) showed that enhancing a summarizer with prosodic information leads to more accurate and informed results. In this work we extend the model proposed by Chen and Metze (2012c), where a random walk is performed on a lexico-topical graph structure to yield summaries. They exploited intra- and inter- speaker relationships through partial topic shar- ing for judging the importance of utterances in the context of multi-party meetings. This paper, on the other hand, enriches the underlying graph struc- ture with prosodic information, rather than lexico- topical knowledge, to model speaker states and emotions. Also different from Maskey and Hirschberg (2005), we model the multimedia document struc- ture as a graph, which allows for flexibility as well as expressive power in representation. This graph structure provides the easy incorporation of targeted features into the …

Content truncated to 500 words