YNU-HPCC at SemEval-2022 Task 8: Transformer-based Ensemble Model for Multilingual News Article Similarity

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1215 - 1220 July 14-15, 2022 ©2022 Association for Computational Linguistics YNU-HPCC at SemEval-2022 Task 8: Transformer-based Ensemble Model for Multilingual News Article Similarity Zihan Nai, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Kunming, China {wangjin,xjzhang}@ynu.edu.cn Abstract This paper describes the system submitted by our team (YNU-HPCC) to SemEval-2022 Task 8: Multilingual news article similarity. This task requires participants to develop a system which could evaluate the similarity between multilingual news article pairs. We propose an approach that relies on Transformers to com- pute the similarity between pairs of news. We tried different models namely BERT, ALBERT, ELECTRA, RoBERTa, M-BERT and Com- pared their results. At last, we chose M-BERT as our System, which has achieved the best Pearson Correlation Coefficient score of 0.738. 1 Introduction In the field of Natural Language Processing (NLP), how to measure the similarity of two texts has been a topic of interest among researchers for decades. Text similarity measures play an increasingly im- portant role in text related research and applications in tasks such as information retrieval, text classifi- cation, document clustering, topic detection, topic tracking, questions generation, question answer- ing, essay scoring, short answer scoring, machine translation, text summarization and others(Gomaa, Fahmy, et al. 2013). It is widely known that text is a high-dimensional semantic space, hence how to abstractly decompose it so that we can quantify its similarity from a mathematical point of view has become the focus for many researchers. There are three methods to measure text similarity: one is the traditional method based on keyword matching, such as N-gram similarity; the second is to map the text to the vector space, and then use the co- sine similarity and other methods; the third is the method of deep learning, such as the deep learn- ing semantic matching model DSSM based on user click data, ConvNet based on convolutional neu- ral network, and the current state-of-art Siamese LSTM and other methods. However, since the in- troduction of bidirectional encoder representations from transformers (BERT)(Devlin, Chang, Lee, and Toutanova 2018), the accuracy and training efficiency in both text classification and sequence labeling have reached new heights. SemEval 2022 Task 8 is a multilingual news ar- ticle similarity task(X. Chen, Zeynali, Camargo, Flöck, Gaffney, Grabowicz, Hale, Jurgens, and Samory 2022). There are mainly 3 difficulties in the task compared with regular text similarity task: (1) the task is interested in the real world- happenings covered in the news articles, not their style of writing, political spin, tone, or any other more subjective design. Therefore, the system built by participant should neglect the subjective part of the text and focus on objective part only;(2) there were over six different languages in both training and test dataset, and some of test data set were composed of multilingual text pair to test the mul- tilingual ability of participating system; (3) the test dataset contains new languages which have never appeared …

Content truncated to 500 words