BeamSeg: A Joint Model for Multi-Document Segmentation and Topic Identification

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 582–592 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 582 BeamSeg: a Joint Model for Multi-Document Segmentation and Topic Identification Pedro Mota Carnegie Mellon University Pittsburgh, PA, USA INESC-ID Instituto Superior T´ecnico Lisboa, Portugal pjdrm@ist.utl.pt Maxine Eskenazi Carnegie Mellon University Pittsburgh, PA, USA max@cs.cmu.edu Lu´ısa Coheur INESC-ID Instituto Superior T´ecnico Lisboa, Portugal lcoheur@l2f.inesc-id.pt Abstract We propose BeamSeg, a joint model for seg- mentation and topic identification of docu- ments from the same domain. The model as- sumes that lexical cohesion can be observed across documents, meaning that segments de- scribing the same topic use a similar lexical distribution over the vocabulary. The model implements lexical cohesion in an unsuper- vised Bayesian setting by drawing from the same language model segments with the same topic. Contrary to previous approaches, we as- sume that language models are not indepen- dent, since the vocabulary changes in consec- utive segments are expected to be smooth and not abrupt. We achieve this by using a dy- namic Dirichlet prior that takes into account data contributions from other topics. BeamSeg also models segment length properties of doc- uments based on modality (textbooks, slides, etc.). The evaluation is carried out in three datasets. In two of them, improvements of up to 4.8% and 7.3% are obtained in the segmen- tation and topic identifications tasks, indicat- ing that both tasks should be jointly modeled. 1 Introduction Documents exhibit a content organization that ag- gregates related text passages in topically coher- ent segments. Understanding the document struc- ture at the segment level enables efficient content navigation. This has become more relevant with the number of available documents on the Web. The current information landscape allows access to documents describing the same subject, provid- ing alternative views or complementary informa- tion. This is advantageous in a variety of scenar- ios. For example, students have at their disposal several learning materials and might need to find a particular topic segment that best suits their learn- ing needs. Finding such documents is an easy task since search engines are capable of returning doc- uments conveying related information. However, if search engines are effective in retrieving these documents, the task of putting them into a co- herent picture remains a challenge (Shahaf et al., 2012). Automatically finding document segments – text segmentation – and identifying which ones discuss the same topic – topic identification – ad- dresses this issue (Jeong and Titov, 2010). Text segmentation and topic identification have been used as intermediate steps in a variety of nat- ural language processing tasks, including summa- rization (Radev et al., 2004), opinion mining (Mu- rakami et al., 2009), semantic and information re- trieval (Purver, 2011; Amoualian et al., 2017). The improvements they brought spurred research in text segmentation. Invariably, all works resort to the lexical cohesion theory (Halliday and Hasan, 1976), which postulates that discourse structure is correlated to the use of cohesive vocabulary. Thus, segments can be identified by …

Content truncated to 500 words