Modeling Structural Similarities between Documents for Coherence Assessment with Graph Convolutional Networks

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7792–7808 July 9-14, 2023 ©2023 Association for Computational Linguistics Modeling Structural Similarities between Documents for Coherence Assessment with Graph Convolutional Networks Wei Liu1, Xiyan Fu2, Michael Strube1 1Heidelberg Institute for Theoretical Studies gGmbH 2Institute for Computational Linguistics, Heidelberg University wei.liu@h-its.org, fu@cl.uni-heidelberg.de, michael.strube@h-its.org Abstract Coherence is an important aspect of text quality, and various approaches have been applied to co- herence modeling. However, existing methods solely focus on a single document’s coherence patterns, ignoring the underlying correlation be- tween documents. We investigate a GCN-based coherence model that is capable of capturing structural similarities between documents. Our model first creates a graph structure for each document, from where we mine different sub- graph patterns. We then construct a hetero- geneous graph for the training corpus, connect- ing documents based on their shared subgraphs. Finally, a GCN is applied to the heterogeneous graph to model the connectivity relationships. We evaluate our method on two tasks, assess- ing discourse coherence and automated essay scoring. Results show that our GCN-based model outperforms all baselines, achieving a new state-of-the-art on both tasks. 1 Introduction Coherence describes the relationship between sen- tences that makes a group of sentences logically connected rather than just a random collection of them (Jurafsky and Martin, 2021). It is an impor- tant aspect of text quality (McNamara et al., 2010), and its modeling has been applied in many down- stream tasks, including summarization (Parveen et al., 2015; Wu and Hu, 2018), dialogue genera- tion (Mesgar et al., 2020; Xu et al., 2021), machine translation (Xiong et al., 2019; Tan et al., 2019) and document-level text generation (Wang et al., 2021; Diao et al., 2021). Given the importance of the task, there is a long line of methods proposed for coherence modeling. Previous models leverage linguistic features to solve the problem. For example, entity grid-based methods (Barzilay and Lapata, 2005; Elsner and Charniak, 2011) capture the entity transition be- tween adjacent sentences of a text to model local !! !" !# !$ "# "! "$ Figure 1: An example of structurally similar documents connected through subgraphs, in which d2 is more simi- lar in structure to d1 than d3. di denotes the i-th docu- ment and gj denotes the j-th subgraph pattern, an edge will be built between them if the sentence graph of di contains subgraph gj. coherence; in contrast, graph-based models (Guin- audeau and Strube, 2013; Mesgar and Strube, 2015) measure coherence using the entity-graph of a doc- ument. Recently, neural network models (Li and Hovy, 2014; Li and Jurafsky, 2017; Mesgar and Strube, 2018; Xu et al., 2019b; Farag and Yan- nakoudakis, 2019; Moon et al., 2019; Jeon and Strube, 2020a,b; Mesgar et al., 2021) have been ap- plied to the task due to their strength in representa- tion learning and feature combination. Those mod- els learn a document’s representation from word embeddings or pre-trained language models, giv- ing significantly better performance than …

Content truncated to 500 words