Graph-based Coherence Modeling For Assessing Readability

Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (*SEM 2015), pages 309–318, Denver, Colorado, June 4–5, 2015. Graph-based Coherence Modeling For Assessing Readability Mohsen Mesgar and Michael Strube Heidelberg Institute for Theoretical Studies gGmbH Schloss-Wolfsbrunnenweg 35 69118 Heidelberg, Germany (mohsen.mesgar|michael.strube)@h-its.org Abstract Readability depends on many factors rang- ing from shallow features like word length to semantic ones like coherence. We intro- duce novel graph-based coherence features based on frequent subgraphs and compare their ability to assess the readability of Wall Street Journal articles. In contrast to Pitler and Nenkova (2008) some of our graph-based features are significantly correlated with hu- man judgments. We outperform Pitler and Nenkova (2008) in the readability ranking task by more than 5% accuracy thus establishing a new state-of-the-art on this dataset. 1 Introduction Readability depends on many factors which enable readers to process a text. These factors can be used by readability assessment methods to quantify the difficulty of text understanding. Possible applica- tions of readability assessment are automatic text summarization and simplification systems. Measur- ing readability can also be used in question answer- ing and knowledge extraction systems to prune texts with low readability (Kate et al., 2010). Many different text features have been used to assess readability. They include shallow features (Flesch, 1948; Kincaid et al., 1975), language modeling features (Si and Callan, 2001; Collins- Thompson and Callan, 2004), syntactic features (Schwarm and Ostendorf, 2005) and text flow or coherence (Barzilay and Lapata, 2008; Pitler and Nenkova, 2008). In a coherent text each sentence has some connections with other sentences. Al- though these local connections make the text more readable, the corresponding coherence features used in Pitler and Nenkova (2008) (Section 2) are not strongly correlated with human judgments. The main goal of this paper is to introduce novel graph-based coherence features for assessing read- ability. To achieve this goal, we use the entity graph coherence model by Guinaudeau and Strube (2013) (Section 3.1.1) and follow two ideas. The first main idea is to use a graph representation of rhetorical re- lations between sentences of a text (Section 3.1.2) and to merge the entity graph and the rhetorical graph (Section 3.1.3). Hence we enrich the entity graph and consequently consider the distribution of two aspects of coherence (i.e. entities and discourse relations) simultaneously. The second main idea is to apply subgraph mining algorithms to find frequent subgraphs (i.e. patterns) in texts (Section 3.2). Sub- graph mining has been successfully applied to other tasks, e.g. image processing (Nowozin et al., 2007) and language modeling (Biemann et al., 2012). We hypothesize that text coherence correlates with fre- quent subgraphs (vaguely reminding us of coherence patterns (Daneˇs, 1974)) and that the mined patterns are good predictors for readability ratings. Our study is novel in introducing new and infor- mative graph-based coherence features. We examine the predictive power of these feature in two experi- ments: first, readability rating prediction, and sec- ond, ranking texts according to the readability (Sec- tion 5). 309 s1 s2 …

Content truncated to 500 words