GoodNewsEveryone: A Corpus of News Headlines Annotated with Emotions, Semantic Roles, and Reader Perception

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1554–1566 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1554 GoodNewsEveryone: A Corpus of News Headlines Annotated with Emotions, Semantic Roles, and Reader Perception Laura Bostan, Evgeny Kim, Roman Klinger Institut f¨ur Maschinelle Sprachverarbeitung, Universit¨at Stuttgart Pfaffenwaldring 5b, 70569 Stuttgart, Germany {laura.bostan, evgeny.kim, roman.klinger}@ims.uni-stuttgart.de Abstract Most research on emotion analysis from text focuses on the task of emotion classification or emotion intensity regression. Fewer works address emotions as a phenomenon to be tackled with structured learning, which can be explained by the lack of relevant datasets. We fill this gap by releasing a dataset of 5000 English news headlines annotated via crowdsourcing with their associated emotions, the corresponding emotion experiencers and textual cues, related emotion causes and targets, as well as the reader’s perception of the emotion of the headline. This annotation task is comparably challenging, given the large number of classes and roles to be identified. We therefore propose a multiphase annotation procedure in which we first find relevant instances with emotional content and then annotate the more fine-grained aspects. Finally, we develop a baseline for the task of automatic prediction of semantic role structures and discuss the results. The corpus we release enables further research on emotion classification, emotion intensity prediction, emotion cause detection, and supports further qualitative studies. Keywords: emotion, structured learning, role labeling 1. Introduction Research in emotion analysis from text focuses on mapping words, sentences, or documents to emotion categories based on the models of Ekman (1992) or Plutchik (2001), which propose the emotion classes of joy, sadness, anger, fear, trust, disgust, anticipation and surprise. Emotion analysis has been applied to a variety of tasks including large scale social media mining (Stieglitz and Dang-Xuan, 2013), literature analysis (Reagan et al., 2016; Kim and Klinger, 2019), lyrics and music analysis (Mihalcea and Strapparava, 2012; Dodds and Danforth, 2010), and the analysis of the development of emotions over time (Hellrich et al., 2019). There are at least two types of questions that cannot yet be answered by these emotion analysis systems. Firstly, such systems do not often explicitly model the perspective of understanding the written discourse (reader, writer, or the text’s point of view). For example, the headline “Djokovic happy to carry on cruising” (Herman, 2019) contains an explicit mention of joy carried by the word “happy”. How- ever, it may evoke different emotions in a reader (e. g., when the reader is a supporter of Roger Federer), and the same applies to the author of the headline. To the best of our knowledge, only one work considers this point (Buechel and Hahn, 2017c). Secondly, the structure that can be associ- ated with the emotion description in text is not uncovered. Questions like “Who feels a particular emotion?” or “What causes that emotion?” remain unaddressed. There has been almost no work in this direction, with only a few exceptions in English (Kim and Klinger, 2018; Mohammad et al., 2014) …

Content truncated to 500 words