Topical Coherence in LDA-based Models through Induced Segmentation

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1799–1809 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1165 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1799–1809 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1165 Topical Coherence in LDA-based Models through Induced Segmentation Hesam Amoualian Univ. Grenoble Alps, CNRS, Grenoble INP - LIG hesam.amoualian@imag.fr Wei Lu Singapore University of Technology and Design luwei@sutd.edu.sg Eric Gaussier Univ. Grenoble Alps, CNRS, Grenoble INP - LIG eric.gaussier@imag.fr Georgios Balikas Univ. Grenoble Alps, CNRS, Grenoble INP - LIG georgios.balikas@imag.fr Massih-Reza Amini Univ. Grenoble Alps, CNRS, Grenoble INP - LIG massih-reza.amini@imag.fr Marianne Clausel Univ. Grenoble Alps, CNRS, Grenoble INP - LJK marianne.clausel@imag.fr Abstract This paper presents an LDA-based model that generates topically coherent segments within documents by jointly segmenting documents and assigning topics to their words. The coherence between topics is ensured through a copula, binding the top- ics associated to the words of a segment. In addition, this model relies on both doc- ument and segment specific topic distribu- tions so as to capture fine grained differ- ences in topic assignments. We show that the proposed model naturally encompasses other state-of-the-art LDA-based models de- signed for similar tasks. Furthermore, our experiments, conducted on six different publicly available datasets, show the effec- tiveness of our model in terms of perplexity, Normalized Pointwise Mutual Information, which captures the coherence between the generated topics, and the Micro F1 measure for text classification. 1 Introduction Since the seminal works of Hofmann (1999) and Blei et al. (2003), there have been several develop- ments in probabilistic topic models. Many exten- sions have indeed been proposed for different ap- plications, including ad-hoc information retrieval (Wei and Croft, 2006), clustering search results (Zeng et al., 2004) and driving faceted browsing (Mimno and McCallum, 2007). In most of these studies, the initial exchangeability assumptions of PLSA and LDA, stipulating that words within a document are interdependent, has led to incoherent topic assignments within semantically meaningful text units, even though the importance of having topically coherent phrases is generally admitted (Griffiths et al., 2005). More recently, (Balikas et al., 2016b) has shown that binding topics, so as to obtain more coherent topic assignments, within such text segments as noun phrases improves the performance (e.g. in terms of perplexity) of LDA- based models. The question nevertheless remains as to which segmentation one should rely on. Furthermore, text segments can refer to topics that are barely present in other parts of the doc- ument. For example, the segment “the Kurdish regional capital” in the sentence1 “A thousand protesters took to the main street in Erbil, the Kur- dish regional capital, to condemn a new law requir- ing all public demonstrations to have government permits.” refers to geography in a document that is mainly devoted to politics. Relying on a single topic distribution, as done in most previous studies including (Balikas et al., 2016b), may …

Content truncated to 500 words