HFL at SemEval-2022 Task 8: A Linguistics-inspired Regression Model with Data Augmentation for Multilingual News Similarity

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1114 - 1120 July 14-15, 2022 ©2022 Association for Computational Linguistics HFL at SemEval-2022 Task 8: A Linguistics-inspired Regression Model with Data Augmentation for Multilingual News Similarity Zihang Xu†, Ziqing Yang†, Yiming Cui‡†, Zhigang Chen† †State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China ‡Research Center for SCIR, Harbin Institute of Technology, Harbin, China †{zhxu13,zqyang5,ymcui,zgchen}@iflytek.com ‡ymcui@ir.hit.edu.cn Abstract This paper describes our system designed for SemEval-2022 Task 8: Multilingual News Ar- ticle Similarity. We proposed a linguistics- inspired model trained with a few task-specific strategies. The main techniques of our system are: 1) data augmentation, 2) multi-label loss, 3) adapted R-Drop, 4) samples reconstruction with the head-tail combination. We also present a brief analysis of some negative methods like two-tower architecture. Our system ranked 1st on the leaderboard while achieving a Pearson’s Correlation Coefficient of 0.818 on the official evaluation set. 1 Introduction In Task 8 (Chen et al., 2022), we are expected to assess the similarity of pairs of multilingual news articles as shown in Table 1. Ten different lan- guages are covered in this task, including Spanish, Italian, German, English, Chinese, Arabic, Polish, French, Turkish and Russian. Task 8 emphasizes more the events themselves described in the news rather than the style of writing or other subjective characteristics. Therefore, it is beneficial to im- prove the quality of clustering of news articles and to explore similar news coverage across different outlets or regions. The foundation model (Bommasani et al., 2021) we choose is XLM-RoBERTa (XLM-R) (Conneau et al., 2019) which has been proved to be a powerful multilingual pre-trained language model compared with other models like mBERT (Devlin et al., 2018) and it can process all the languages existing in Task 8. Based on that, a great variety of strategies have been tested along with our exploration like data augmentation (DA), head-tail combination, multi- label loss, adapted R-Drop, etc. Through this task, we realized the importance of data quality and efficient training schemes in such a cross-lingual setting. By struggling to improve the richness of the data and find out what meth- ods are effective when training such a similarity assessment model, our system1 ranked 1st in this competition. 2 Background 2.1 Dataset Description There are 4,964 samples with 8 language pairs in the training set and the test set contains 4,593 sam- ples in 18 different language pairs, the details of which are presented in Table 2. Due to some inac- cessible URLs, the training set is slightly smaller than it should be (22 samples missing in total). The similarity scores of pairs of articles in the provided dataset are rated on a 4-point scale (be- tween 1 and 4) from most to least similar in 7 sub- dimensions, including Geography, Entities, Time, Narrative, Overall, Style and Tone (an example is provided in Appendix). However, only the pre- dictions for Overall will be used to evaluate the performance of our systems. 2.2 Related Work Research on text …

Content truncated to 500 words