Team QUST at SemEval-2023 Task 3: A Comprehensive Study of Monolingual and Multilingual Approaches for Detecting Online News Genre, Framing and Persuasion Techniques

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 300–306 July 13-14, 2023 ©2023 Association for Computational Linguistics Team QUST at SemEval-2023 Task 3: A Comprehensive Study of Monolingual and Multilingual Approaches for Detecting Online News Genre, Framing and Persuasion Techniques Ye Jiang Qingdao University of Science and Technology, China ye.jiang@qust.edu.cn Abstract This paper describes the participation of team QUST in the SemEval2023 task 3. The mono- lingual models are first evaluated with the under-sampling of the majority classes in the early stage of the task. Then, the pre-trained multilingual model is fine-tuned with a com- bination of the class weights and the sample weights. Two different fine-tuning strategies, the task-agnostic and the task-dependent, are further investigated. All experiments are con- ducted under the 10-fold cross-validation, the multilingual approaches are superior to the monolingual ones. The submitted system1 achieves the second best in Italian and Spanish (zero-shot) in subtask-1. 1 Introduction Task 3 (Piskorski et al., 2023) expects the partici- pants to develop algorithms to automatically detect the news genre, framing and persuasion techniques in a multilingual setup as shown in Table 1. Six dif- ferent languages are covered in this task, including English, French, German, Italian, Polish and Rus- sian. In addition, three surprise languages, Spanish, Greek and Georgian, are also included in the fi- nal test phase for conducting a zero-shot learning scenario. One of the major challenges of this task is that the model must have the ability to tackle the issues of imbalanced and insufficient training data. In subtask-1, the number of samples for labels ‘re- porting’, ‘opinion’ and ‘satire’ are 269, 878 and 87 respectively. With only few ‘satire’ samples relative to other majority classes such as ‘opinion’, the training model will easily overfit on majority classes and not learn enough from minority ones. For example, many batches will have no ‘satire’ samples, so the gradients will be less informative during the training phase and result in poor per- formances in predicting the minority class. Mean- 1https://github.com/zgjiangtoby/SemEval2023_QUST while, previous studies (Leite et al., 2020; Zhang et al., 2021) demonstrate that the scale of training data could influence the model performance sig- nificantly. However, the total training samples are 1234 in subtask-1, which is not enough to effec- tively train a deep learning model from scratch. To address the above issues, traditional machine learning and deep learning methods are first evalu- ated with the under-sampling of the majority class in a monolingual setting. Then, the pre-trained multilingual model is fine-tuned to learn the cross- language features between languages. Specifically, fine-tuning is a common approach for utilizing the pre-trained language model (PLM) on many downstream tasks. It typically replaces the origi- nal output layer with a new task-specific layer and then fine-tunes the complete model. To model the features representation of news articles across dif- ferent languages, the XLM-RoBERTa (XLM-R) (Conneau et al., 2020) is fine-tuned since it can processes all the languages existing in Task 3, and typically outperforms other models such as …

Content truncated to 500 words