Automatic Assessment of Conceptual Text Complexity Using Knowledge Graphs

Proceedings of the 27th International Conference on Computational Linguistics, pages 318–330 Santa Fe, New Mexico, USA, August 20-26, 2018. 318 Automatic Assessment of Conceptual Text Complexity Using Knowledge Graphs Sanja ˇStajner and Ioana Hulpus¸ Data and Web Science Group University of Mannheim, Germany {sanja,ioana}@informatik.uni-mannheim.de Abstract Complexity of texts is usually assessed only at the lexical and syntactic levels. Although it is known that conceptual complexity plays a significant role in text understanding, no attempts have been made at assessing it automatically. We propose to automatically estimate the conceptual complexity of texts by exploiting a number of graph-based measures on a large knowledge base. By using a high-quality language learners corpus for English, we show that graph-based measures of individual text concepts, as well as the way they relate to each other in the knowledge graph, have a high discriminative power when distinguishing between two versions of the same text. Furthermore, when used as features in a binary classification task aiming to choose the simpler of two versions of the same text, our measures achieve high performance even in a default setup. 1 Introduction A text can be complex on various levels. It can contain many infrequent words that are unknown to the reader (lexical complexity). It can contain long sentences with difficult syntactic structures or syntactic structures unknown to the reader (syntactic complexity), especially in the case of children or non-native language learners. A much less studied level of text complexity is the conceptual or semantic level, which accounts for the amount of background knowledge required to understand the meaning of the text. While many methods have been proposed so far for automatic assessment of lexical and syntactic text complexity (Vajjala and Meurers, 2014; Vajjala and Meurers, 2015; Ambati et al., 2016), automatic assessment of conceptual text complexity has never been attempted so far. Similarly, in the task of automatic text simplification, only two systems attempted at simplifying the overall text structure by eliminating irrelevant pieces of information (Narayan and Gardent, 2014; ˇStajner and Glavaˇs, 2017). Texts that are conceptually complex tend to contain difficult and abstract concepts which are not easy to relate to each other, or they mention numerous entities which are not closely related, requiring substantial background knowledge. Kintsch and van Dijk (1978) distinguish between the micro- and macrostructure of text semantics. The former takes into account the individual concepts and their local relations, while the latter considers their organization at a global level. Conceptual complexity is reflected in both those components and thus conceptual simplification must also be performed on both levels. Conceptual complexity is especially important for low-knowledge readers, who might have problems accessing their prior knowledge (Denton et al., 2015; McNamara et al., 2012). They benefit particularly from texts that require less inferences, and not so much from simplifications at the linguistic level (Arf´e et al., 2017). The amount of inferences that readers need to make in order to fill in the gaps in text coherence, and the effort needed to understand the text …

Content truncated to 500 words