SkoltechNLP at SemEval-2022 Task 8: Multilingual News Article Similarity via Exploration of News Texts to Vector Representations

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1136 - 1144 July 14-15, 2022 ©2022 Association for Computational Linguistics SkoltechNLP at SemEval-2022 Task 8: Multilingual News Article Similarity via Exploration of News Texts to Vector Representations Mikhail Kuimov Daryna Dementieva Alexander Panchenko Skolkovo Institute of Science and Technology {Mikhail.Kuimov, Daryna.Dementieva, A.Panchenko}@skoltech.ru Abstract This paper describes our contribution to Se- mEval 2022 Task 8 on Multilingual News Ar- ticle Similarity. The aim was to test com- pletely different approaches and distinguish the best performing. That is why we’ve consid- ered systems based on Transformer-based en- coders, NER-based, and NLI-based methods (and their combination with SVO dependency triplets representation). The results prove that Transformer models produce the best scores. However, there is space for research and ap- proaches that give not yet comparable but more interpretable results. 1 Introduction The SemEval 2022 Task 8 competition (Chen et al., 2022) aims to develop systems that identify mul- tilingual news articles that provide similar infor- mation. This is a document-level similarity task in the applied domain of news articles, rating them pairwise on a 4-point scale from most to least sim- ilar. The alikeness of news is measured in such a sense: how similar are them in geography, time, shared entities, and shared narratives. The devel- oped approaches for solving the task can be applied to several different real-world tasks. The first one is the clustering of news. A lot of news-providing companies want thou- sands of articles from different publishers on one topic to be combined in a single page showing the news. Another task is Fake News Detection. This task has become extremely important to solve lately. Different articles on the same news story can be compared to find contradictions in facts and details. This can be an indicator that the story is fake like it is shown in (Dementieva and Panchenko, 2021). 2 Related Work In (Montalvo et al., 2007), authors extract PERSON, ORGANIZATION and LOCATION named entities (NE) and compose a vector for each of the cate- gory with the help of Levenshtein distance function and TF-IDF weighting function, which combines Term Frequency (TF) and Inverse Document Fre- quency (IDF). These 3 vectors are compared to 3 corresponding vectors of the second news with cosine distance. Obtained scores are combined with the set of IF-THEN rules. In (Rahimi et al., 2019) the approach for cross-lingual transfer is proposed which is evaluated on the Named Entity Recognition task. Dialogue competition (Gusev and Smurov, 2021) on Russian news clustering has produced many promising methods which could be adopted to the multilingual case. Most of them, like (Sergei et al., 2021; Glazkova, 2021), are the variations of fine-tuning the transformer models and making ensembles. In (Martín et al., 2021), the authors developed the pipeline for checking the news on veracity. They compare embeddings of the news under con- sideration with ones from the database, using co- sine distance. Then they take the most similar news found and apply Natural Language …

Content truncated to 500 words