STaCK: Sentence Ordering with Temporal Commonsense Knowledge

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8676–8686 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8676 STaCK: Sentence Ordering with Temporal Commonsense Knowledge Deepanway Ghosal†, Navonil Majumder†, Rada Mihalcea△, Soujanya Poria† † Singapore University of Technology and Design, Singapore △ University of Michigan, USA deepanway_ghosal@mymail.sutd.edu.sg {navonil_majumder, sporia}@sutd.edu.sg mihalcea@umich.edu Abstract Sentence order prediction is the task of finding the correct order of sentences in a randomly ordered document. Correctly ordering the sen- tences requires an understanding of coherence with respect to the chronological sequence of events described in the text. Document- level contextual understanding and common- sense knowledge centered around these events are often essential in uncovering this coher- ence and predicting the exact chronological order. In this paper, we introduce STaCK — a framework based on graph neural net- works and temporal commonsense knowledge to model global information and predict the relative order of sentences. Our graph net- work accumulates temporal evidence using knowledge of ‘past’ and ‘future’ and formu- lates sentence ordering as a constrained edge classification problem. We report results on five different datasets, and empirically show that the proposed method is naturally suit- able for order prediction, thus demonstrating the role of temporal commonsense knowledge. The implementation of this work is available at: https://github.com/declare-lab/ sentence-ordering. 1 Introduction Coherence is an essential quality for any natural language text (Halliday and Hasan, 1976). Cor- rect ordering of sentences is a necessary attribute of coherence. As such, there has been much re- search in correct sentence order detection due to its application to various down-stream tasks, such as, question answering (Yu et al., 2018), multi- document summarization (Barzilay et al., 2002), automated content addition to text (Mostafazadeh et al., 2016), text generation (Mostafazadeh et al., 2016), and others. It also has potential applica- tions in the evaluation of the quality of machine- generated documents. Existing approaches to sen- tence order prediction can be broadly classified into two categories: (1) sequence generation methods, and (2) pair-wise methods (Zhu et al., 2021; Prab- humoye et al., 2020). While the former considers tagging the entire sequence, the latter takes one sentence pair at a time and predicts their relative order. Pair-wise methods ignore the importance of document level global information, i.e., while pre- dicting the relative order of two sentences (si, sj), other sentences sk from the same document do not play any role. Global document information Jennifer has her final exam tomorrow. She got so stressed, she pulled an all-nighter. She went into class the next day, weary. Her teacher postponed the test for next week. Jennifer felt bittersweet about it. Jennifer will miss her friends after college. She has a great job lined up after graduation. That’s the end of her college term. Jennifer felt bittersweet about it. Jennifer has her final exam tomorrow. Document A Document B Placements of highlighted sentences differ with document-level information Happens after Happens after Figure 1: Position of two sentences differ based on the dissimilar contextual utterances; the …

Content truncated to 500 words