Indexing Spoken Documents with Hierarchical Semantic Structures: Semantic Tree-to-string Alignment Models

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 509–517, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Indexing Spoken Documents with Hierarchical Semantic Structures: Semantic Tree-to-string Alignment Models Xiaodan Zhu & Colin Cherry Institute for Information Technology National Research Council Canada {Xiaodan.Zhu,Colin.Cherry}@nrc-cnrc.gc.ca Gerald Penn Department of Computer Science University of Toronto gpenn@cs.toronto.edu Abstract This paper addresses a semantic tree-to- string alignment problem: indexing spo- ken documents with known hierarchical semantic structures, with the goal to help index and access such archives. We pro- pose and study a number of alignment models of different modeling capabilities and time complexities to provide a com- prehensive understanding of these unsu- pervised models and hence the problem it- self. 1 Introduction The inherent difficulties in efficiently accessing spoken documents raise the need for ways to better organize such archives. Such a need parallels with the consistently increasing demand for and avail- ability of audio content on web pages and other digital media, which, in turn, should come as no surprise, with speech being one of the most basic, most natural forms of human communication. When intended to be read, written documents are almost always presented as more than unin- terrupted text strings; e.g., indicative structures such as section/subsection headings and tables-of- contents are standard constituents created manu- ally to help readers, whereas structures of this kind are rarely aligned with spoken documents, which has raised little concern—in most time of history, speech has not been ready for navigation, un- til very recently, when recording, delivering, and even automatic transcription were possible. Navigating audio documents is often inherently much more difficult than browsing text. An ob- vious solution, in relying on human beings’ abil- ity of reading text, is to conduct a speech-to-text conversion through ASR, which in turn raises a new set of problems to be considered. First, the convenience and efficiency of reading transcripts are affected by errors produced in transcription channels, though if the goal is only to browse the most salient parts, recognition errors in ex- cerpts can be reduced by considering ASR con- fidence (Xie and Liu, 2010; Hori and Furui, 2003; Zechner and Waibel, 2000) and the quality of ex- cerpts can be improved from various perspectives (Zhang et al., 2010; Xie and Liu, 2010; Zhu et al., 2009; Murray, 2008; Zhu and Penn, 2006; Maskey and Hirschberg, 2005). Even if transcrip- tion quality were not a problem, browsing lengthy transcripts is not straightforward, since, as men- tioned above, indicative browsing structures are barely manually created for and aligned with spo- ken documents. Ideally, such semantic structures should be inferred directly from the spoken doc- uments themselves, but this is known to be diffi- cult even for written texts, which are often more linguistically well-formed and less noisy than au- tomatically transcribed text. This paper studies a less ambitious problem: we align an already- existing hierarchical browsing structure, e.g., the electronic slides of lecture recordings, with the se- quential transcripts of the corresponding …

Content truncated to 500 words