EMBEDDIA at SemEval-2022 Task 8: Investigating Sentence, Image, and Knowledge Graph Representations for Multilingual News Article Similarity

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1107 - 1113 July 14-15, 2022 ©2022 Association for Computational Linguistics EMBEDDIA at SemEval-2022 Task 8: Investigating Sentence, Image, and Knowledge Graph Representations for Multilingual News Article Similarity Elaine Zosa∗ University of Helsinki Helsinki, Finland elaine.zosa@helsinki.fi Emanuela Boros∗ University of La Rochelle La Rochelle, France emanuela.boros@univ-lr.fr Boshko Koloski∗ Jožef Stefan Institute Jožef Stefan Institute IPS Ljubljana, Slovenia boshko.koloski@ijs.si Lidia Pivovarova University of Helsinki Helsinki, Finland lidia.pivovarova@helsinki.fi Abstract In this paper, we present the participation of the EMBEDDIA team in the SemEval-2022 Task 8 (Multilingual News Article Similarity). We cover several techniques and propose differ- ent methods for finding the multilingual news article similarity by exploring the dataset in its entirety. We take advantage of the textual content of the articles, the provided metadata (e.g., titles, keywords, topics), the translated articles, the images (those that were available), and knowledge graph-based representations for entities and relations present in the articles. We, then, compute the semantic similarity between the different features and predict through re- gression the similarity scores. Our findings show that, while our proposed methods ob- tained promising results, exploiting the seman- tic textual similarity with sentence represen- tations is unbeatable. Finally, in the official SemEval-2022 Task 8, we ranked fifth in the overall team ranking cross-lingual results, and second in the English-only results. 1 Introduction Detecting news stories related to a single theme and combining them into news clusters has been an in- creasing interest in the creation of news aggregators that consolidate thousands of articles from differ- ent publishers and websites (Pranji´c et al., 2020). Tracking similarity of news coverage between dif- ferent outlets or regions has also been urgent and challenging. For example, whether previously with Ebola or recently with the COVID-19 pandemic, monitoring and containment of infectious disease outbreaks has remained a key component of public health strategy to contain the diseases. The ability ∗* Equal contribution from all the authors. to track disease outbreaks in an accurate manner is critical in the deployment of efficient intervention measures. As such reports may not only be in En- glish, there is also a need for effective multilingual systems. Hence, recent research has been focused on the area of identifying similarities between doc- uments, phrases, stories, etc. Semantic textual similarity (STS) deals with de- termining how similar two groups of sentences are by measuring their semantic similarity. Over the years, several solutions were proposed to assess STS. The most general approach is pre-training on massive datasets before fine-tuning on subsequent downstream tasks (Jiang et al., 2020; Raffel et al., 2019; Lan et al., 2019; Yang et al., 2019; Liu et al., 2019; Sanh et al., 2019). Other works considered finding the similarity by classifying texts using BERT-based models (Devlin et al., 2019) with a pair of sentences packed together as input (Yang et al., 2019; Liu et al., 2019; Sanh et al., 2019; Wang et al., 2019). The SemEval-2022 Task 8 (Multilingual News Article Similarity) (Chen et al., …

Content truncated to 500 words