A Neural CRF-based Hierarchical Approach for Linear Text Segmentation

Findings of the Association for Computational Linguistics: EACL 2023, pages 883–893 May 2-6, 2023 ©2023 Association for Computational Linguistics A Neural CRF-based Hierarchical Approach for Linear Text Segmentation Inderjeet Nair, Aparna Garimella, Balaji Vasan Srinivasan, Natwar Modani, Niyati Chhaya, Srikrishna Karanam, Sumit Shekhar Adobe Research, India {inair,garimell,balsrini,nmodani, nchhaya,skaranam,sushekha}@adobe.com Abstract We consider the problem of segmenting unfor- matted text and transcripts linearly based on their topical structure. While prior approaches explicitly train to predict segment boundaries, we propose to address this task by inferring the hierarchical segmentation structure associ- ated with the input text. For this purpose, we present a data curation strategy to obtain the hierarchical segmentation structure annotations for over 700K Wikipedia articles. We then pro- pose the first supervised approach to generate hierarchical segmentation structures for given text based on a neural conditional random field (CRF) that explicitly models the statistical de- pendencies between nodes and their constituent children. We introduce a novel data augmen- tation scheme as part of our model training, which involves sampling a variety of node ag- gregations, permutations, and removals, all of which help capture fine-grained and coarse top- ical shifts in the data and improve model per- formance. Extensive experiments show that our model outperforms or achieves competi- tive performance when compared to previous state-of-the-art algorithms in the following set- tings: rich-resource, cross-domain transferabil- ity, few-shot supervision, and segmentation when topic label annotations are provided. 1 Introduction Text segmentation (Hearst, 1997; Choi, 2000), an important task in information retrieval, is defined as the process of dividing unstructured text into topically coherent segments. Because it recovers topical structure from unformatted text, it can be used as a pre-processing step for several down- stream tasks such as text summarization (Mitra et al., 1997), question answering (Oh et al., 2007) and discourse analysis (Van Dijk, 1982). Most prior works on text segmentation (Hearst, 1997; Choi, 2000; Koshorek et al., 2018) attempted to address this task by explicitly predicting the segment boundaries, with the assumption that any given text can be decomposed into contiguous, non- overlapping, indivisible segments, based on topical themes. The discourse segmentation theory (Grosz and Sidner, 1986), however, asserts that the out- come may not always be strictly decompositional, i.e., a segment may have sub-segments within it, and segments may overlap with each other. Fol- lowing this theory, we hypothesize that explicitly training to infer the hierarchical topic structure of the underlying text leads to better linear segmenta- tion, as it forces the models to examine text at mul- tiple levels to extract coarse-grained to fine-grained topical segments. Further, this allows inference of linear segments of varying granularity that can be used for various downstream applications. Previous works on hierarchical segmentation are largely unsupervised (Eisenstein, 2009; Simon et al., 2015) due to the unavailability of large la- belled datasets with hierarchical structure informa- tion. In this paper, we propose to leverage the hi- erarchical structures in a supervised manner, given the superior performances of supervised models across several language processing tasks (Mikolov et …

Content truncated to 500 words