CoCo: A Tool for Automatically Assessing Conceptual Complexity of Texts

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7179–7186 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7179 CoCo: A Tool for Automatically Assessing Conceptual Complexity of Texts Sanja ˇStajner1, Sergiu Nisioi2, Ioana Hulpus¸3 1ReadableAI, Cologne, Germany 2Human Language Technologies Research Center, University of Bucharest, Romania 3Data and Web Science Group, University of Mannheim, Germany sanja.stajner@readableai.com, sergiu.nisioi@gmail.com, ioana@informatik.uni-mannheim.de Abstract Traditional text complexity assessment usually takes into account only syntactic and lexical text complexity. The task of automatic assessment of conceptual text complexity, important for maintaining reader’s interest and text adaptation for struggling readers, has only been proposed recently. In this paper, we present CoCo - a tool for automatic assessment of conceptual text complexity, based on using the current state-of-the-art unsupervised approach. We make the code and API freely available for research purposes, and describe the code and the possibility for its personalization and adaptation in details. We compare the current implementation with the state of the art, discussing the influence of the choice of entity linker on the performances of the tool. Finally, we present results obtained on two widely used text simplification corpora, discussing the full potential of the tool. Keywords: conceptual complexity, spreading activation, text analysis tool 1. Introduction To actively engage readers in the text they are reading and maintain their interest in the topic, texts needs to be on an adequate level for the reader in terms of its lexical, syntac- tic, as well as its conceptual complexity (McNamara et al., 2006). Lexical complexity refers to lexical choices used in the text, e.g. the frequency of words in everyday life, rich- ness of vocabulary, use of some specific terminology or for- eign words, etc. Syntactic complexity refers to the length of the sentences and their syntactic structures, e.g. the number of subordinate or coordinate clauses, use of passive voice, unusual sentence structures, depth of the syntactic tree, etc. Conceptual complexity refers to the model of reading com- prehension proposed by Kintsch and van Dijk (1978). Ac- cording to this model, the reader needs to understand both individual propositions and concepts in the text, as well as their relations, in order to make a coherent story and fully understand the text. Text difficulty could thus be seen as the amount of gaps in the text coherence, and the effort re- quired by the reader to repair them by inference making (Arf´e et al., 2017). Assessment of text complexity has a long history in edu- cation and psycholinguistic research (DuBay, 2004). It is used to assess the reading level required by the students to understand the materials in the schoolbooks, complex- ity of army manuals, or complexity of documents available for general public, such as healthcare documents for ex- ample. Only in the second half of the last century, over 200 readability formulae were proposed with the aim of measuring text complexity (DuBay, 2004). Despite many criticisms of their simplicity (DuBay, 2004), some of them gained high …

Content truncated to 500 words