Topic Segmentation with a Structured Topic Model

Proceedings of NAACL-HLT 2013, pages 190–200, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Topic Segmentation with a Structured Topic Model Lan Du Department of Computing Macquarie University Sydney, Australia lan.du@mq.edu.au Wray Buntine Canberra Research Lab National ICT Australia Canberra, Australia wray.buntine@nicta.com.au Mark Johnson Department of Computing Macquarie University Sydney, Australia mark.johnson@mq.edu.au Abstract We present a new hierarchical Bayesian model for unsupervised topic segmentation. This new model integrates a point-wise boundary sam- pling algorithm used in Bayesian segmenta- tion into a structured topic model that can cap- ture a simple hierarchical topic structure latent in documents. We develop an MCMC infer- ence algorithm to split/merge segment(s). Ex- perimental results show that our model out- performs previous unsupervised segmentation methods using only lexical information on Choi’s datasets and two meeting transcripts and has performance comparable to those pre- vious methods on two written datasets. 1 Introduction Documents are usually comprised of topically co- herent text segments, each of which contains some number of text passages (e.g., sentences or para- graphs) (Salton et al., 1996). Within each topically coherent segment, one would expect that the word usage demonstrates more consistent lexical distri- butions (known as lexical cohesion (Eisenstein and Barzilay, 2008)) than that across segments. A linear partition of texts into topic segments may reveal in- formation about, for example, themes of segments and the overall thematic structure of the text, and can subsequently be useful for text analysis tasks, such as information retrieval (e.g., passage retrieval (Salton et al., 1996)), document summarisation and discourse analysis (Galley et al., 2003). In this paper we consider how to automatically find a topic segmentation. It involves identifying the most prominent topic changes in a sequence of text passages, and splits those passages into a sequence of topically coherent segments (Hearst, 1997; Beeferman et al., 1999). This task can be cast as an unsupervised machine learning problem: plac- ing topic boundaries in unannotated text. Although a variety of cues in text can be used for topic segmentation, such as cue phases (Beeferman et al., 1999; Reynar, 1999; Eisenstein and Barzi- lay, 2008)) and discourse information (Galley et al., 2003), in this paper, we focus on lexical cohesion and use it as the primary cue in developing an un- supervised segmentation model. The effectiveness of lexical cohesion has been demonstrated by Text- Tiling (Hearst, 1997), c99 (Choi, 2000), MinCut (Malioutov and Barzilay, 2006), PLDA (Purver et al., 2006), Bayesseg (Eisenstein and Barzilay, 2008), TopicTiling (Riedl and Biemann, 2012), etc. Our work uses recent progress in hierarchi- cal topic modelling with non-parametric Bayesian methods (Du et al., 2010; Chen et al., 2011; Du et al., 2012a), and is based on Bayesian segmentation methods (Goldwater et al., 2009; Purver et al., 2006; Eisenstein and Barzilay, 2008) using topic mod- els. This can also be viewed as a multi-topic exten- sion of hierarchical Bayesian segmentation (Eisen- stein, 2009), although our use of hierarchies is used to improve the performance of linear segmentation, rather than develop hierarchical segmentation. …

Content truncated to 500 words