Lessons Learnt from Linear Text Segmentation: a Fair Comparison of Architectural and Sentence Encoding Strategies for Successful Segmentation

Proceedings of Recent Advances in Natural Language Processing, pages 408–418 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_046 408 Lessons Learnt from Linear Text Segmentation: a Fair Comparison of Architectural and Sentence Encoding Strategies for Successful Segmentation Iacopo Ghinassi Queen Mary University of London / London, UK i.ghinassi@qmul.ac.uk Lin Wang Queen Mary University of London / London, UK lin.wang@qmul.ac.uk Chris Newell BBC R&D / London, UK chris.newell@bbc.co.uk Matthew Purver Queen Mary University of London / London, UK Institut Joˇzef Stefan / Ljubljana, Slovenia m.purver@qmul.ac.uk Abstract Recent works on linear text segmentation have shown new state-of-the-art results nearly every year. Most times, however, these recent ad- vances include a variety of different elements which makes it difficult to evaluate which indi- vidual components of the proposed methods bring about improvements for the task and, more generally, what actually works for linear text segmentation. Moreover, evaluating text segmentation is notoriously difficult and the use of a metric such as Pk, which is widely used in existing literature, presents specific problems that complicates a fair comparison between segmentation models. In this work, then, we draw from a number of existing works to assess which is the state-of-the-art in linear text segmentation, investigating what architec- tures and features work best for the task. For doing so, we present three models representa- tive of a variety of approaches, we compare them to existing methods and we inspect ele- ments composing them, so as to give a more complete picture of which technique is more successful and why that might be the case. At the same time, we highlight a specific feature of Pk which can bias the results and we report our results using different settings, so as to give future literature a more comprehensive set of baseline results for future developments. We then hope that this work can serve as a solid foundation to foster research in the area, over- coming task-specific difficulties such as evalua- tion setting and providing new state-of-the-art results1. 1code available at: https://github.com/Ighina/NSE- TopicSegmentation 1 Introduction Linear text segmentation, also known as topic seg- mentation, is a well known problem in natural lan- guage processing, and the first step for a number of downstream applications. The task consists in the automatic segmentation of a text into topically coherent units and this has many use cases: a long transcript from a news show, e.g., could be divided into single news stories so as to help an end user in retrieving more relevant and specific information (Reynar, 1999) or a long article could be divided into subsections to aid its reading (Hearst, 1997). Recent works have presented a series of advance- ments in the field, from which a number of conclu- sions could be drawn, such as the fact that Trans- former architectures work better than traditional re- current models (Lo et al., 2021) and that fine-tuned LLMs need no additional contextual information to perform the task (Lee et al., 2023). The results of different recent works, however, can be contradictory and not pointing towards …

Content truncated to 500 words