Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderings

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2407–2417 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2407 Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderings Shaopeng Lai1, Ante Wang1, Fandong Meng2, Jie Zhou2, Yubin Ge3, Jiali Zeng4, Junfeng Yao1, Degen Huang5 and Jinsong Su1,6∗ 1Xiamen University, China 2Pattern Recognition Center, WeChat AI, Tencent Inc, China 3University of Illinois at Urbana-Champaign, USA 4Tencent Cloud Xiaowei, China 5Dalian University of Technology, China 6Pengcheng Laboratory, China splai@stu.xmu.edu.cn, fandongmeng@tencent.com, jssu@xmu.edu.cn Abstract Dominant sentence ordering models can be classified into pairwise ordering models and set-to-sequence models. However, there is lit- tle attempt to combine these two types of mod- els, which inituitively possess complementary advantages. In this paper, we propose a nov- el sentence ordering framework which intro- duces two classifiers to make better use of pair- wise orderings for graph-based sentence order- ing (Yin et al., 2019, 2021). Specially, giv- en an initial sentence-entity graph, we first introduce a graph-based classifier to predict pairwise orderings between linked sentences. Then, in an iterative manner, based on the graph updated by previously predicted high- confident pairwise orderings, another classifier is used to predict the remaining uncertain pair- wise orderings. At last, we adapt a GRN-based sentence ordering model (Yin et al., 2019, 2021) on the basis of final graph. Experiments on five commonly-used datasets demonstrate the effectiveness and generality of our model. Particularly, when equipped with BERT (De- vlin et al., 2019) and FHDecoder (Yin et al., 2020), our model achieves state-of-the-art per- formance. Our code is available at https:// github.com/DeepLearnXMU/IRSEG. 1 Introduction With the rapid development and increasing applica- tions of natural language processing (NLP), model- ing text coherence has become a significant task, s- ince it can provide beneficial information for under- standing, evaluating and generating multi-sentence texts. As an important subtask, sentence order- ing aims at recovering unordered sentences back to naturally coherent paragraphs. It is required to deal with logic and syntactic consistency, and has increasingly attracted attention due to its wide ap- plications on several tasks such as text generation (Konstas and Lapata, 2012; Holtzman et al., 2018) ∗ Corresponding author and extractive summarization (Barzilay et al., 2002; Nallapati et al., 2012). Recently, inspired by the great success of deep learning in other NLP tasks, researchers have re- sorted to neural sentence ordering models, which can be classified into: pairwise ordering models (Chen et al., 2016; Agrawal et al., 2016; Li and Jurafsky, 2017; Moon et al., 2019; Kumar et al., 2020; Prabhumoye et al., 2020; Zhu et al., 2021) and set-to-sequence models (Gong et al., 2016; N- guyen and Joty, 2017; Logeswaran et al., 2018; Mohiuddin et al., 2018; Cui et al., 2018; Yin et al., 2019; Oh et al., 2019; Yin et al., 2020; Cui et al., 2020; Yin et al., 2021). Generally, the former pre- dicts the relative orderings between pairwise sen- tences, which are then leveraged to produce the final ordered sentence sequence. Its advantage lies in the lightweight pairwise …

Content truncated to 500 words