Is Everything in Order? A Simple Way to Order Sentences

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10769–10779 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 10769 Is Everything in Order? A Simple Way to Order Sentences Somnath Basu Roy Chowdhury1∗ Faeze Brahman2∗ Snigdha Chaturvedi1 1UNC Chapel Hill, 2University of California, Santa Cruz {somnath, snigdha}@cs.unc.edu fbrahman@ucsc.edu Abstract The task of organizing a shuffled set of sen- tences into a coherent text has been used to evaluate a machine’s understanding of causal and temporal relations. We formulate the sentence ordering task as a conditional text- to-marker generation problem. We present Reorder-BART (RE-BART) that leverages a pre-trained Transformer-based model to iden- tify a coherent order for a given set of shuf- fled sentences. The model takes a set of shuf- fled sentences with sentence-specific markers as input and generates a sequence of position markers of the sentences in the ordered text. RE-BART achieves the state-of-the-art perfor- mance across 7 datasets in Perfect Match Ra- tio (PMR) and Kendall’s tau (τ). We perform evaluations in a zero-shot setting, showcasing that our model is able to generalize well across other datasets. We additionally perform sev- eral experiments to understand the functioning and limitations of our framework. 1 Introduction Constructing coherent text requires an understand- ing of entities, events, and their relationships. Auto- matically understanding such relationships among nearby sentences in a multi-sentence text has been a longstanding challenge in NLP. Sentence ordering task was proposed to test the ability of automatic models to reconstruct a coher- ent text given a set of shuffled sentences (Barzilay and Lapata, 2005). Coherence modeling has wide applications in natural language generation like extraction-based multi-document summarization (Barzilay and Elhadad, 2002; Galanis et al., 2012; Nallapati et al., 2017), retrieval dependent QA (Yu et al., 2018; Liu et al., 2018) and concept-to-text generation (Schwartz et al., 2017). Earlier studies on coherence modeling and sen- tence ordering focused on exploiting different cate- gories of features like coreference clues (Elsner and ∗Authors contributed equally. I packed my raincoat. The forecast called for rainy. It never rained. The weather is never predictable. Instead it started to snow. 1 2 3 4 5 The forecast called for rainy. I packed my raincoat. It never rained. Instead it started to snow. The weather is never predictable. 2 1 3 5 4 Shuffled Input Ordered Output Figure 1: An example of the sentence ordering task. The goal is to reorder a set of shuffled sentences (left) into a coherent sequence of text (right). Charniak, 2008), entity grids (Lapata and Barzilay, 2005; Barzilay and Lapata, 2005), named-entity categories (Elsner and Charniak, 2011), and syn- tactic features (Louis and Nenkova, 2012) among others. With the advent of deep learning, re- searchers leveraged distributed sentence representa- tions learned through recurrent neural networks (Li and Hovy, 2014). Recent works adopted ranking- based algorithms to solve the task (Chen et al., 2016; Kumar et al., 2020; Prabhumoye et al., 2020). In this paper, we present RE-BART (for Reorder- BART) to solve the sentence ordering as …

Content truncated to 500 words