ITNLP2022 at SemEval-2022 Task 8: Pre-trained Model with Data Augmentation and Voting for Multilingual News Similarity

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1184 - 1189 July 14-15, 2022 ©2022 Association for Computational Linguistics ITNLP2022 at SemEval-2022 Task 8: Pre-trained Model with Data Augmentation and Voting for Multilingual News Similarity Zhongan Chen∗, Weiwei Chen∗, Yunlong Sun∗, Hongqing Xu∗, Shuzhe Zhou∗, Bohan Chen ∗, Chengjie Sun, Yuanchao Liu Intelligence Technology and Natural Language Processing Lab, School of Computer Science and Technology, Harbin Institute of Technology {zachen, wwchen, ylsun, hqxu, szzhou, bhchen}@insun.hit.edu.cn {cjsun, lyc}@insun.hit.edu.cn Abstract This article introduces a system to solve the SemEval 2022 Task 8: Multilingual News Arti- cle Similarity. The task focuses on the con- sistency of events reported in two news ar- ticles. The system consists of a pre-trained model(e.g., INFOXLM and XLM-RoBERTa) to extract multilingual news features, following fully-connected networks to measure the simi- larity. In addition, data augmentation and Ten Folds Voting are used to enhance the model. Our final submitted model is an ensemble of three base models, with a Pearson value of 0.784 on the test dataset. 1 Introduction The task (Chen et al., 2022) aims to design a sys- tem that can find the similarities of multi-language news articles. The task focuses on the consistency of actual events reported in two news articles, but not the subjective factors such as writing style or political factors. Geographical location, time, com- mon entity, and common narrative were used to judge similarity. Consistency is measured by a float value between [1,4]. The lower the value is, the more likely the two news are reporting the same thing. The metric is the Pearson Coefficient of the predicted and ground truth values. Challenge of this task: 1. Need to understand ev- ery aspect of a news event: what happened, where and when, who was involved, and why and how it happened 2. Make the writing style and common phrases clear because some unnecessary content can be misleading. 3. Some information about the event is hard to get, such as the time, location, and description of the event. We use the pre-trained language model XLM- Roberta and INFOXLM to fine-tune. Specifically, we splice the available information of two news articles (exact method will be described in Chapter 1*These authors contributed equally. 3), input it into the pre-trained model, and then transmit the output vector to the fully connected layer of downstream tasks. For the original fine- tuning task, we try to use various techniques to make the program run faster and work better. The techniques include 1. Freezing the lower layers of the pre-trained model, means not updating their parameters during training. 2. Data Augmentation. Use translation software to translate the original news text to expand the data. 3. Divide the training set into 10 parts. the ten folds voting was adopted to make full use of the data set. Our code is available on github1. 2 Related Work 2.1 Background The input of the task is the content of two news, and the output is the Overall label of them. …

Content truncated to 500 words