SemEval 2022 Task 10: Structured Sentiment Analysis

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1280 - 1295 July 14-15, 2022 ©2022 Association for Computational Linguistics SemEval 2022 Task 10: Structured Sentiment Analysis Jeremy Barnes1, Laura Oberländer2, Enrica Troiano2, Andrey Kutuzov3 Jan Buchmannn4, Rodrigo Agerri1, Lilja Øvrelid3, Erik Velldal3 1HiTZ Center - Ixa, University of the Basque Country UPV/EHU 2IMS, University of Stuttgart 3LTG, University of Oslo 4Technical University of Darmstadt {jeremy.barnes, rodrigo.agerri}@ehu.eus {laura.oberlaender, enrica.troiano}@ims.uni-stuttgart.de {andreku, liljao, erikve}@ifi.uio.no buchmann@ukp.informatik.tu-darmstadt.de Abstract In this paper, we introduce the first SemEval shared task on Structured Sentiment Analysis, for which participants are required to predict all sentiment graphs in a text, where a single sentiment graph is composed of a sentiment holder, target, expression and polarity. This new shared task includes two subtracks (mono- lingual and cross-lingual) with seven datasets available in five languages, namely Norwegian, Catalan, Basque, Spanish and English. Partici- pants submitted their predictions on a held-out test set and were evaluated on Sentiment Graph F1. Overall, the task received over 200 submis- sions from 32 participating teams. We present the results of the 16 teams that provided system descriptions and our own expanded analysis of the test predictions. 1 Introduction Affective computing is a fundamental step to- wards enabling human computer interaction (Pi- card, 1997), as human communication is filled with affective content which conveys a speaker’s private state, i.e. their current mood, their emotional expe- riences, or their attitude towards a certain object of conversation. Along with emotion detection, sen- timent analysis (Pang et al., 2002; Turney, 2002; Wiebe et al., 2005) is an important stepping stone towards this goal. On a more practical level, being able to automatically determine what people think about an idea, product, or policy is of interest to companies, governments, and private citizens. In this paper, we describe the SemEval-2022 shared task on Structured Sentiment Analysis, which can be thought of as an information ex- traction problem in which one attempts to find all of the opinion tuples O = Oi, . . . , On in a text. Each opinion Oi is a tuple (h, t, e, p) where h is a holder who expresses a polarity p towards a tar- get t through a sentiment expression e, implicitly defining pairwise relationships between elements of the same tuple. Liu (2012) argues that all of these elements are essential to fully resolve the sentiment analysis problem. Although early anno- tation efforts in sentiment analysis annotated for fine-grained sentiment (Wiebe et al., 2005; Toprak et al., 2010), most research on modeling sentiment focuses either on a variety of sub-tasks which avoid performing the full task, e.g. targeted (Hu and Liu, 2004), aspect-based (Pontiki et al., 2014), or end-2-end sentiment (Wang et al., 2016), or in- stead relies on simplified and idealized tasks, e.g. sentence-level binary polarity classification (Pang et al., 2002). We argue that the division of fine-grained sen- timent into these sub-tasks has become counter- productive, as reported experiments are often not sensitive to whether a given addition to …

Content truncated to 500 words