SemEval-2023 Task 3: Detecting the Category, the Framing, and the Persuasion Techniques in Online News in a Multi-lingual Setup

SemEval-2023 Task 3: Detecting the Category, the Framing, and the Persuasion Techniques in Online News in a Multi-lingual Setup Jakub Piskorski1, Nicolas Stefanovitch2, Giovanni Da San Martino3, Preslav Nakov4 1Institute of Computer Science, Polish Academy of Science, Poland jpiskorski@gmail.com 2European Commission Joint Research Centre, Italy nicolas.stefanovitch@ec.europa.eu 3Department of Mathematics, University of Padova, Italy dasan@math.unipd.it 4Mohamed bin Zayed University of Artificial Intelligence, UAE preslav.nakov@mbzuai.ac.ae Abstract We describe SemEval-2023 task 3 on Detect- ing the Category, the Framing, and the Per- suasion Techniques in Online News in a Multi- lingual Setup: the dataset, the task organization process, the evaluation setup, the results, and the participating systems. The task focused on news articles in nine languages (six known to the participants upfront: English, French, Ger- man, Italian, Polish, and Russian), and three ad- ditional ones revealed to the participants at the testing phase: Spanish, Greek, and Georgian). The task featured three subtasks: (1) determin- ing the genre of the article (opinion, reporting, or satire), (2) identifying one or more frames used in an article from a pool of 14 generic frames, and (3) identify the persuasion tech- niques used in each paragraph of the article, using a taxonomy of 23 persuasion techniques. This was a very popular task: a total of 181 teams registered to participate, and 41 even- tually made an official submission on the test set. 1 Introduction The widespread use of Internet and the advances in Internet-related technologies paved the way to eas- ily create direct communication channels between information producers and consumers, potentially leaving the latter exposed to manipulative, propa- gandistic, and deceptive content. Given the poten- tially huge audience that can be reached through online channels, major public events and debates re- volving around relevant topics could be influenced as a result. Therefore, there is an ever-growing need to develop automated tools supporting experts in analysing the news ecosystem and identifying large-scale manipulation attempts, and facilitating the study of how different events, global topics, and policies are being embraced by media in var- ious countries, in order to carry out cross-country analysis and to gather knowledge on the ways how media informs public opinion, i.e., what aspects are being highlighted and linked to a topic, what pros and cons are mentioned, the way opinions are conveyed, and what rhetorical devices, i.e., logical fallacies and appeal to emotions, are used to sup- port flawed argumentation, potentially leading to manipulation. To foster research in this direction, there have been several shared tasks asking to detect the use of specific propaganda techniques in text, as well as the specific span of each instance. This includes the NLP4IF-2019 shared task on Fine-Grained Propa- ganda Detection (Da San Martino et al., 2019), SemEval-2020 task 11 on Detection of Persua- sion Techniques in News Articles (Da San Martino et al., 2020a), SemEval-2021 task 6 on Detection of Persuasion Techniques in Texts and Images (Dim- itrov et al., 2021b), and WANLP-2022 Shared Task on Propaganda Detection in Arabic (Alam et al., 2022). Our task is …

Content truncated to 500 words