Chapter Ordering in Novels

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3838 - 3848 December 7-11, 2022 ©2022 Association for Computational Linguistics Chapter Ordering in Novels Allen Kim, Steven Skiena Department of Computer Science, Stony Brook University, NY, USA {allekim,skiena}@cs.stonybrook.edu Abstract Understanding narrative flow and text coher- ence in long-form documents (novels) remains an open problem in NLP. To gain insight, we explore the task of chapter ordering, recon- structing the original order of chapters in novel given a random permutation of the text. This can be seen as extending the well-known sen- tence ordering task to vastly larger documents: our task deals with over 9,000 novels with an average of twenty chapters each, versus stan- dard sentence ordering datasets averaging only 5-8 sentences. We formulate the task of recon- structing order as a constraint solving problem, using minimum feedback arc set and traveling salesman problem optimization criteria, where the weights of the graph are generated based on models for character occurrences and chap- ter boundary detection, using relational chapter scores derived from RoBERTa. Our best meth- ods yield a Spearman correlation of 0.59 on this novel and challenging task, substantially above baseline. 1 Introduction Novels are a fundamental long-form medium for storytelling, requiring understanding of narrative flow for full comprehension of the text. Although neural network-based language models demon- strate amazing performance in comprehending short text windows, the problems of understanding long-form narrative texts such as summarization remain largely open. A major challenge in research on long-form nar- ration is the cost of annotation: the mere act of reading a novel generally requires a 10-15 hour commitment on the part of the annotator, making it clearly cost prohibitive to obtain full annotations of a significant corpus of novels. Although published summaries are available for up to a few hundred popular books, annotation cost is the rate limiting step for work in this area (Wu et al., 2021). In this paper, we propose a new task that gets to the heart of narrative flow analysis for novels, with- out the need for human annotation. Most novels are partitioned into chapters as a means of providing thematic breakpoints for the reader. Randomly per- muting the order of these chapters obviously ruins the coherence of the narrative flow of the underly- ing story. Here we propose the new task of chapter ordering: reconstructing the original order of the book given an input random chapter permutation of the text. This task is clearly related to the task of sen- tence ordering (Lapata, 2003; Barzilay and Lapata, 2005, 2008), reconstructing a set of sentences in a tiny story as to maximize their coherence. While there is a large volume of prior work on sentence ordering, to our knowledge, this is the first paper to address chapter ordering. The natural approaches for the sentence task cannot be readily applied to chapter order. First, chapters in novels usually contain several hundred sentences, which cannot easily be encoded with the neural models applied to sentence …

Content truncated to 500 words