Dimensions of Quality: Contrasting Stylistic vs. Semantic Features for Modelling Literary Quality in 9,000 Novels

Proceedings of Recent Advances in Natural Language Processing, pages 739–747 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_080 739 Dimensions of Quality: Contrasting Stylistic vs. Semantic Features for Modelling Literary Quality in 9,000 Novels Pascale Feldkamp Moreira School of Communication and Culture Aarhus University, Denmark pascale.moreira@cc.au.dk Yuri Bizzoni Center for Humanities Computing Aarhus University, Denmark yuri.bizzoni@cc.au.dk Abstract In computational literary studies, the challeng- ing task of predicting quality or reader apprecia- tion of narrative texts is confounded by volatile definitions of quality and the vast feature space that may be considered in modeling. In this paper, we explore two different types of fea- ture sets: stylistic features on one hand, and semantic and sentiment features on the other. We conduct experiments on a corpus of 9,089 English language literary novels published in the 19th and 20th century, using GoodReads’ ratings as a proxy for reader appreciation. Ex- amining the potential of both approaches, we find that some types of books are more pre- dictable in one model than in the other, which may indicate that texts have different promi- nent characteristics (i.a., stylistic complexity, narrative progression at the sentiment-level). 1 Introduction Defining literary quality or reader appreciation is a complex challenge for quantitative literary studies due to the the heterogeneous nature of narrative texts, and the complexity of mechanisms of judge- ments and standards in the literary field. While recent studies demonstrate that literary quality ap- pears above chance at the scale of large numbers, and that both text-extrinsic and text-intrinsic fea- tures systematically impact sales figures and reader judgements (Wang et al., 2019; Lassen et al., 2022; Koolen et al., 2020; Bizzoni et al., 2022a; Mahar- jan et al., 2017), the question of how these features interact, and what metrics can be used to validate them, remains open. The challenge lies not merely in modeling literary quality, but in selecting which features to include in a model, while ensuring a de- gree of interpretability. In this study, we examine two different sets of textual features for modelling literary quality: stylistic and syntactic characteris- tics vs. narrative and semantic features based on sentiment analysis and word-category profiling. 2 Related works Generally, we may distinguish two types of feature- sets used to model literary quality: stylistic fea- tures (the “how” of writing) and those that cap- ture deeper structures and content (the “what” of writing). Previous studies of literary quality have predominantly relied on stylistic features, such as sentence-length, lexical richness or redundancy (Koolen et al., 2020; Maharjan et al., 2017), syn- tactic complexity (Zedelius et al., 2019), or n-gram frequencies (Koolen et al., 2020). More recent works have tested the effect of al- ternative features, such as sentiment analysis on reader experience (Drobot, 2013; Kim and Klinger, 2018; Brooke et al., 2015; Jockers, 2017; Reagan et al., 2016). Studies relying on sentiment analysis usually draw scores from lexica (Islam et al., 2020) or human annotations (Mohammad and Turney, 2013), to outline the sentiment arcs of narrative texts (Jockers, 2017), and have shown a …

Content truncated to 500 words