ACCEPT at SemEval-2023 Task 3: An Ensemble-based Approach to Multilingual Framing Detection

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1347–1358 July 13-14, 2023 ©2023 Association for Computational Linguistics ACCEPT at SemEval-2023 Task 3: An Ensemble-based Approach to Multilingual Framing Detection Philipp Heinisch Bielefeld University pheinisch@techfak.uni-bielefeld.de Moritz Plenz Heidelberg University plenz@cl.uni-heidelberg.de Anette Frank Heidelberg University frank@cl.uni-heidelberg.de Philipp Cimiano Bielefeld University cimiano@techfak.uni-bielefeld.de Abstract This paper describes the system and experi- mental results of an ensemble-based approach to multilingual framing detection for the sub- mission of the ACCEPT team to the SemEval- 2023 Task 3 on Framing Detection (Subtask 2). The approach is based on an ensemble that com- bines three different methods: a classifier based on large language models, a classifier based on static word embeddings, and an approach that uses external commonsense knowledge graphs, in particular, ConceptNet. The results of the three classification heads are aggregated into an overall prediction for each frame class. Our best submission yielded a micro F1-score of 50.69% (rank 10) and a macro F1-score of 50.20% (rank 3) for English articles. Our ex- perimental results show that static word embed- dings and knowledge graphs are useful compo- nents for frame detection, while the ensemble of all three methods combines the strengths of our three proposed methods. Through sys- tem ablations, we show that the commonsense- guided knowledge graphs are the outperform- ing method for many languages. 1 Introduction The SemEval 2023 Task 3 by Piskorski et al. (2023) (Subtask 2) consisted in detecting different types of frames in newspaper articles in six different lan- guages (English, French, German, Italian, Polish, and Russian). Beyond these seen languages, three more languages were added in the testing phase (Spanish, Greek, and Georgian). Different framings represent different perspec- tives on one and the same reported event. Being able to categorize them helps to cluster such news articles and guides the analyses of different as- pects (Card et al., 2015). In the particular task at hand, the applied generic MediaFrames set (Boyd- stun et al., 2014) categorizes articles into 14 frame classes: “Economic”, “Capacity and resources”, “Morality”, “Fairness and equality”, “Legality, con- stitutionality and jurisprudence”, “Policy prescrip- tion and evaluation”, “Crime and punishment”, “Se- curity and defense”, “Health and safety”, “Quality of life”, “Cultural identity”, “Public opinion”, “Po- litical” and “External regulation and reputation”. In this paper, we describe the system and exper- imental results for submitting team ACCEPT to the shared task. Our approach consists of an en- semble of three different methods, each relying on a fully connected classification layer on top of i) a large language model (LM), ii) static word em- beddings and iii) a Graph Neural Network (GNN) architecture exploiting a commonsense knowledge graph (CSKG), ConceptNet in particular. To han- dle inputs in multiple languages, as required by the task, we explore language-specific and language- agnostic methods, and determine the best config- uration for each language. As an interesting re- sult, we found that the overall best configuration is language-specific by using commonsense knowl- edge graphs for inference, while LMs do not per- …

Content truncated to 500 words